Credibility of In Silico Trial Technologies-A Theoretical Framing

An open source statistical web application for validation and analysis of virtual cohorts

The conventional approach to developing medical treatments and medical devices usually covers pre-clinical and in-vitro investigations, in-vivo animal studies and clinical trials with humans. In-silico trials and virtual cohorts present a promising avenue for addressing the challenges inherent in clinical research and improving its efficiency. Despite considerable advancements in the field of in-silico trials, several notable gaps and challenges still need to be addressed, one is the limited availability of open and user-friendly statistical tools to support the specific analysis of virtual cohorts and in-silico trials. In the EU-Horizon funded project SIMCor we have developed a web application, providing a R-statistical environment supporting the validation of virtual cohorts and the application of validated cohorts for in-silico trials. It provides a practical platform for validating cohorts and has implemented existing statistical techniques that can be applied to compare virtual cohorts with real datasets. It is fully open, generic and menu driven and provides user guidance and help ( https://github.com/ecrin-github/SIMCor , https://zenodo.org/records/14718597 ).The tool has been developed according to specified user requirements and has been extensively tested and validated. Important next steps are to gain more experience with the tool in other domains and research environments and to extend its functionality.

Advancing in Silico Clinical Trials for Regulatory Adoption and Innovation

The evolution of information and communication technologies has affected all fields of science, including health sciences. However, the rate of technological innovation adoption by the healthcare sector has been historically slow, compared to other industrial sectors. Innovation in computer modeling and simulation approaches has changed the landscape in biomedical applications and biomedicine, paving the way for their potential contribution in reducing, refining, and partially replacing animal and human clinical trials. In Silico Clinical Trials (ISCT) allow the development of virtual populations used in the safety and efficacy testing of new drugs and medical devices. This White Paper presents the current framework for ISCT, the role of in silico medicine research communities, the different perspectives (research, scientific, clinical, regulatory, standardization, data quality, legal and ethical), the barriers, challenges, and opportunities for ISCT adoption. In addition, an overview of successful ISCT projects, market-available platforms, and FDA- approved paradigms, along with their vision, mission and outcomes are presented.

Synthetic Health Data: Real Ethical Promise and Peril

Researchers and practitioners are increasingly using machine-generated synthetic data as a tool for advancing health science and practice, by expanding access to health data while-potentially-mitigating privacy and related ethical concerns around data sharing. While using synthetic data in this way holds promise, we argue that it also raises significant ethical, legal, and policy concerns, including persistent privacy and security problems, accuracy and reliability issues, worries about fairness and bias, and new regulatory challenges. The virtue of synthetic data is often understood to be its detachment from the data subjects whose measurement data is used to generate it. However, we argue that addressing the ethical issues synthetic data raises might require bringing data subjects back into the picture, finding ways that researchers and data subjects can be more meaningfully engaged in the construction and evaluation of datasets and in the creation of institutional safeguards that promote responsible use.

A GENERATIVE APPROACH TO TESTING THE PERFORMANCE OF PHYSIOLOGICAL CONTROL ALGORITHMS

Background

Physiological closed-loop control algorithms play an important role in the development of autonomous medical care systems, a promising area of research that has the potential to deliver healthcare therapies meeting each patient's specific needs. Computational approaches can support the evaluation of physiological closed-loop control algorithms considering various sources of patient variability that they may be presented with.

Method of Approach

In this paper, we present a generative approach to testing the performance of physiological closed-loop control algorithms. This approach exploits a generative physiological model (which consists of stochastic and dynamic components that represent diverse physiological behaviors across a patient population) to generate a select group of virtual subjects. By testing a physiological closed-loop control algorithm against this select group, the approach estimates the distribution of relevant performance metrics in the represented population. We illustrate the promise of this approach by applying it to a practical case study on testing a closed-loop fluid resuscitation control algorithm designed for hemodynamic management.

Results

In this context, we show that the proposed approach can test the algorithm against virtual subjects equipped with a wide range of plausible physiological characteristics and behavior, and that the test results can be used to estimate the distribution of relevant performance metrics in the represented population.

Conclusions

In sum, the generative testing approach may offer a practical, efficient solution for conducting pre-clinical tests on physiological closed-loop control algorithms.

Credibility assessment of in silico clinical trials for medical devices

In silico clinical trials (ISCTs) are an emerging method in modeling and simulation where medical interventions are evaluated using computational models of patients. ISCTs have the potential to provide cost-effective, time-efficient, and ethically favorable alternatives for evaluating the safety and effectiveness of medical devices. However, ensuring the credibility of ISCT results is a significant challenge. This paper aims to identify unique considerations for assessing the credibility of ISCTs and proposes an ISCT credibility assessment workflow based on recently published model assessment frameworks. First, we review various ISCTs described in the literature, carefully selected to showcase the range of methodological options available. These studies cover a wide variety of devices, reasons for conducting ISCTs, patient model generation approaches including subject-specific versus 'synthetic' virtual patients, complexity levels of devices and patient models, incorporation of clinician or clinical outcome models, and methods for integrating ISCT results with real-world clinical trials. We next discuss how verification, validation, and uncertainty quantification apply to ISCTs, considering the range of ISCT approaches identified. Based on our analysis, we then present a hierarchical workflow for assessing ISCT credibility, using a general credibility assessment framework recently published by the FDA's Center for Devices and Radiological Health. Overall, this work aims to promote standardization in ISCTs and contribute to the wider adoption and acceptance of ISCTs as a reliable tool for evaluating medical devices.

Generative artificial intelligence empowers digital twins in drug discovery and clinical trials

INTRODUCTION

The concept of Digital Twins (DTs) translated to drug development and clinical trials describes virtual representations of systems of various complexities, ranging from individual cells to entire humans, and enables in silico simulations and experiments. DTs increase the efficiency of drug discovery and development by digitalizing processes associated with high economic, ethical, or social burden. The impact is multifaceted: DT models sharpen disease understanding, support biomarker discovery and accelerate drug development, thus advancing precision medicine. One way to realize DTs is by generative artificial intelligence (AI), a cutting-edge technology that enables the creation of novel, realistic and complex data with desired properties.

AREAS COVERED

The authors provide a brief introduction to generative AI and describe how it facilitates the modeling of DTs. In addition, they compare existing implementations of generative AI for DTs in drug discovery and clinical trials. Finally, they discuss technical and regulatory challenges that should be addressed before DTs can transform drug discovery and clinical trials.

EXPERT OPINION

The current state of DTs in drug discovery and clinical trials does not exploit the entire power of generative AI yet and is limited to simulation of a small number of characteristics. Nonetheless, generative AI has the potential to transform the field by leveraging recent developments in deep learning and customizing models for the needs of scientists, physicians and patients.

A multiscale orchestrated computational framework to reveal emergent phenomena in neuroblastoma

Neuroblastoma is a complex and aggressive type of cancer that affects children. Current treatments involve a combination of surgery, chemotherapy, radiotherapy, and stem cell transplantation. However, treatment outcomes vary due to the heterogeneous nature of the disease. Computational models have been used to analyse data, simulate biological processes, and predict disease progression and treatment outcomes. While continuum cancer models capture the overall behaviour of tumours, and agent-based models represent the complex behaviour of individual cells, multiscale models represent interactions at different organisational levels, providing a more comprehensive understanding of the system. In 2018, the PRIMAGE consortium was formed to build a cloud-based decision support system for neuroblastoma, including a multi-scale model for patient-specific simulations of disease progression. In this work we have developed this multi-scale model that includes data such as patient's tumour geometry, cellularity, vascularization, genetics and type of chemotherapy treatment, and integrated it into an online platform that runs the simulations on a high-performance computation cluster using Onedata and Kubernetes technologies. This infrastructure will allow clinicians to optimise treatment regimens and reduce the number of costly and time-consuming clinical trials. This manuscript outlines the challenging framework's model architecture, data workflow, hypothesis, and resources employed in its development.

Empirical methods for the validation of time-to-event mathematical models taking into account uncertainty and variability: application to EGFR + lung adenocarcinoma

BACKGROUND

Over the past several decades, metrics have been defined to assess the quality of various types of models and to compare their performance depending on their capacity to explain the variance found in real-life data. However, available validation methods are mostly designed for statistical regressions rather than for mechanistic models. To our knowledge, in the latter case, there are no consensus standards, for instance for the validation of predictions against real-world data given the variability and uncertainty of the data. In this work, we focus on the prediction of time-to-event curves using as an application example a mechanistic model of non-small cell lung cancer. We designed four empirical methods to assess both model performance and reliability of predictions: two methods based on bootstrapped versions of parametric statistical tests: log-rank and combined weighted log-ranks (MaxCombo); and two methods based on bootstrapped prediction intervals, referred to here as raw coverage and the juncture metric. We also introduced the notion of observation time uncertainty to take into consideration the real life delay between the moment when an event happens, and the moment when it is observed and reported.

RESULTS

We highlight the advantages and disadvantages of these methods according to their application context. We have shown that the context of use of the model has an impact on the model validation process. Thanks to the use of several validation metrics we have highlighted the limit of the model to predict the evolution of the disease in the whole population of mutations at the same time, and that it was more efficient with specific predictions in the target mutation populations. The choice and use of a single metric could have led to an erroneous validation of the model and its context of use.

CONCLUSIONS

With this work, we stress the importance of making judicious choices for a metric, and how using a combination of metrics could be more relevant, with the objective of validating a given model and its predictions within a specific context of use. We also show how the reliability of the results depends both on the metric and on the statistical comparisons, and that the conditions of application and the type of available information need to be taken into account to choose the best validation strategy.

A combined experimental and finite element analysis of the human elbow under loads of daily living

Elbow trauma is often accompanied by a loss of independence in daily self-care activities, negatively affecting patients' quality of life. Finite element models can help gaining profound knowledge about native human joint mechanics, which is crucial to adequately restore joint functionality after severe injuries. Therefore, a finite element model of the elbow is required that includes both the radio-capitellar and ulno-trochlear joint and is subjected to loads realistic for activities of daily living. Since no such model has been published, we aim to fill this gap. For comparison, 8 intact cadaveric elbows were subjected to loads of up to 1000 N, after they were placed in an extended position. At each load step, the displacement of the proximal humerus relative to the distal base plate was measured with optical tracking markers and the joint pressure was measured with a pressure mapping sensor. Analogously, eight finite element models were created based on subject-specific CT scans of the corresponding elbow specimens. The CT scans were registered to the positions of tantalum beads in the experiment. The optically measured displacements were applied as boundary conditions. We demonstrated that the workflow can predict the experimental contact pressure distribution with a moderate correlation, the experimental peak pressures in the correct joints and the experimental stiffness with moderate to excellent correlation. The predictions of peak pressure magnitude, contact area and load share on the radius require improvement by precise representation of the cartilage geometry and soft tissues in the model, and proper initial contact in the experiment.

Adapting modeling and simulation credibility standards to computational systems biology

Computational models are increasingly used in high-impact decision making in science, engineering, and medicine. The National Aeronautics and Space Administration (NASA) uses computational models to perform complex experiments that are otherwise prohibitively expensive or require a microgravity environment. Similarly, the Food and Drug Administration (FDA) and European Medicines Agency (EMA) have began accepting models and simulations as forms of evidence for pharmaceutical and medical device approval. It is crucial that computational models meet a standard of credibility when using them in high-stakes decision making. For this reason, institutes including NASA, the FDA, and the EMA have developed standards to promote and assess the credibility of computational models and simulations. However, due to the breadth of models these institutes assess, these credibility standards are mostly qualitative and avoid making specific recommendations. On the other hand, modeling and simulation in systems biology is a narrower domain and several standards are already in place. As systems biology models increase in complexity and influence, the development of a credibility assessment system is crucial. Here we review existing standards in systems biology, credibility standards in other science, engineering, and medical fields, and propose the development of a credibility standard for systems biology models.

Relevant Choices Affecting the Fatigue Analysis of Ni-Ti Endovascular Devices

Ni-Ti alloys are widely used for biomedical applications due to their superelastic properties, which are especially convenient for endovascular devices that require minimally invasive insertion and durable effects, such as peripheral/carotid stents and valve frames. After crimping and deployment, stents undergo millions of cyclic loads imposed by heart/neck/leg movements, causing fatigue failure and device fracture that can lead to possibly severe consequences for the patient. Standard regulations require experimental testing for the preclinical assessment of such devices, which can be coupled with numerical modeling to reduce the time and costs of such campaigns and to obtain more information regarding the local state of stress and strain in the device. In this frame, this review aimed to enlighten the relevant choices that can affect the outcome of the fatigue analysis of Ni-Ti devices, both from experimental and numerical perspectives.

Trabecular bone remodeling in the aging mouse: A micro-multiphysics agent-based in silico model using single-cell mechanomics

Bone remodeling is regulated by the interaction between different cells and tissues across many spatial and temporal scales. Notably, in silico models are regarded as powerful tools to further understand the signaling pathways that regulate this intricate spatial cellular interplay. To this end, we have established a 3D multiscale micro-multiphysics agent-based (micro-MPA) in silico model of trabecular bone remodeling using longitudinal in vivo data from the sixth caudal vertebra (CV6) of PolgA(D257A/D257A) mice, a mouse model of premature aging. Our in silico model includes a variety of cells as single agents and receptor-ligand kinetics, mechanomics, diffusion and decay of cytokines which regulate the cells' behavior. We highlighted its capabilities by simulating trabecular bone remodeling in the CV6 of five mice over 4 weeks and we evaluated the static and dynamic morphometry of the trabecular bone microarchitecture. Based on the progression of the average trabecular bone volume fraction (BV/TV), we identified a configuration of the model parameters to simulate homeostatic trabecular bone remodeling, here named basal. Crucially, we also produced anabolic, anti-anabolic, catabolic and anti-catabolic responses with an increase or decrease by one standard deviation in the levels of osteoprotegerin (OPG), receptor activator of nuclear factor kB ligand (RANKL), and sclerostin (Scl) produced by the osteocytes. Our results showed that changes in the levels of OPG and RANKL were positively and negatively correlated with the BV/TV values after 4 weeks in comparison to basal levels, respectively. Conversely, changes in Scl levels produced small fluctuations in BV/TV in comparison to the basal state. From these results, Scl was deemed to be the main driver of equilibrium while RANKL and OPG were shown to be involved in changes in bone volume fraction with potential relevance for age-related bone features. Ultimately, this micro-MPA model provides valuable insights into how cells respond to their local mechanical environment and can help to identify critical pathways affected by degenerative conditions and ageing.

Model verification tools: a computational framework for verification assessment of mechanistic agent-based models

BACKGROUND

Nowadays, the inception of computer modeling and simulation in life science is a matter of fact. This is one of the reasons why regulatory authorities are open in considering in silico trials evidence for the assessment of safeness and efficacy of medicinal products. In this context, mechanistic Agent-Based Models are increasingly used. Unfortunately, there is still a lack of consensus in the verification assessment of Agent-Based Models for regulatory approval needs. VV&UQ is an ASME standard specifically suited for the verification, validation, and uncertainty quantification of medical devices. However, it can also be adapted for the verification assessment of in silico trials for medicinal products.

RESULTS

Here, we propose a set of automatic tools for the mechanistic Agent-Based Model verification assessment. As a working example, we applied the verification framework to an Agent-Based Model in silico trial used in the COVID-19 context.

CONCLUSIONS

Using the described verification computational workflow allows researchers and practitioners to easily perform verification steps to prove Agent-Based Models robustness and correctness that provide strong evidence for further regulatory requirements.

Validation-based model selection for 13C metabolic flux analysis with uncertain measurement errors

Accurate measurements of metabolic fluxes in living cells are central to metabolism research and metabolic engineering. The gold standard method is model-based metabolic flux analysis (MFA), where fluxes are estimated indirectly from mass isotopomer data with the use of a mathematical model of the metabolic network. A critical step in MFA is model selection: choosing what compartments, metabolites, and reactions to include in the metabolic network model. Model selection is often done informally during the modelling process, based on the same data that is used for model fitting (estimation data). This can lead to either overly complex models (overfitting) or too simple ones (underfitting), in both cases resulting in poor flux estimates. Here, we propose a method for model selection based on independent validation data. We demonstrate in simulation studies that this method consistently chooses the correct model in a way that is independent on errors in measurement uncertainty. This independence is beneficial, since estimating the true magnitude of these errors can be difficult. In contrast, commonly used model selection methods based on the χ²-test choose different model structures depending on the believed measurement uncertainty; this can lead to errors in flux estimates, especially when the magnitude of the error is substantially off. We present a new approach for quantification of prediction uncertainty of mass isotopomer distributions in other labelling experiments, to check for problems with too much or too little novelty in the validation data. Finally, in an isotope tracing study on human mammary epithelial cells, the validation-based model selection method identified pyruvate carboxylase as a key model component. Our results argue that validation-based model selection should be an integral part of MFA model development.

Measuring metabolic reaction fluxes in living cells is difficult, yet important. The gold standard is to label extracellular metabolites with ¹³C, to use mass spectrometry to find out where the ¹³C-atoms ends up, and finally use mathematical modelling to calculate how quickly each reaction must have flowed, for the ¹³C-atoms to end up like that. This measurement thus relies on usage of the right mathematical model, which must be selected among various candidate models. In this manuscript, we present a new way to do this model selection step, utilizing validation data. Using an adopted approach to calculate the uncertainty of model predictions, we identify new validation experiments, which are neither too similar, nor too dissimilar, compared to the previous training data. The model candidate that is best at predicting this new validation data is the one chosen. Tests on simulated data where the true model is known, shows that the validation-based method is robust when the magnitude of the error in the measurement uncertainty is unknown, something that conventional methods are not. This improvement is important since true uncertainties can be difficult to estimate for these data. Finally, we demonstrate how the new method can be used on real data, to identify fluxes and important reactions.

[Enriching in vivo clinical trials with in silico models for orthopedic implants]

Clinical trials are used by the medical device industry to confirm products safety, performance, and clinical benefits. Traditional clinical studies typically follow a limited number of volunteers, which prevents capturing the full breath of patient demographics and implant use. New tools are required to overcome the limitations of traditional trials while fulfilling increasingly demanding regulatory requirements. Computer simulations have the potential to enrich traditional clinical trials with so called in silico clinical trials (ISCT) by providing data on a much broader spectrum of patients, clinical conditions and implant configurations. The historical use of simulation in the orthopedic device industry is described here to explain how it is now technically possible to model virtual populations. We also discuss the multiple benefits of such a translational research approach for the patients, healthcare systems, and manufacturers, but also the challenges to overcome. A more detailed version is available in English [1].

On the Role and Effects of Uncertainties in Cardiovascular in silico Analyses

The assessment of cardiovascular hemodynamics with computational techniques is establishing its fundamental contribution within the world of modern clinics. Great research interest was focused on the aortic vessel. The study of aortic flow, pressure, and stresses is at the basis of the understanding of complex pathologies such as aneurysms. Nevertheless, the computational approaches are still affected by sources of errors and uncertainties. These phenomena occur at different levels of the computational analysis, and they also strongly depend on the type of approach adopted. With the current study, the effect of error sources was characterized for an aortic case. In particular, the geometry of a patient-specific aorta structure was segmented at different phases of a cardiac cycle to be adopted in a computational analysis. Different levels of surface smoothing were imposed to define their influence on the numerical results. After this, three different simulation methods were imposed on the same geometry: a rigid wall computational fluid dynamics (CFD), a moving-wall CFD based on radial basis functions (RBF) CFD, and a fluid-structure interaction (FSI) simulation. The differences of the implemented methods were defined in terms of wall shear stress (WSS) analysis. In particular, for all the cases reported, the systolic WSS and the time-averaged WSS (TAWSS) were defined.

Impact of anatomical reverse remodelling in the design of optimal quadripolar pacing leads: A computational study

Lead position is an important factor in determining response to Cardiac Resynchronization Therapy (CRT) in dyssynchronous heart failure (HF) patients. Multipoint pacing (MPP) enables pacing from multiple electrodes within the same lead, improving the potential outcome for patients. Virtual quadripolar lead designs were evaluated by simulating pacing from all combinations of 1 and 2 electrodes along the lead in each virtual patient from cohorts of HF (n = 24) and simulated reverse remodelled (RR, n = 20) patients. Electrical synchrony was assessed by the time 90% of the ventricular myocardium is activated (AT090). Optimal 1 and 2 electrode pacing configurations for AT090 were combined to identify the 4-electrode lead design that maximised benefits across all patients. LV pacing in the HF cohort in all possible single and double electrode locations reduced AT090 by 14.48 ± 5.01 ms (11.92 ± 3.51%). The major determinant of reduction in activation time was patient anatomy. Pacing with a single optimal lead design reduced AT090 more in the HF cohort than the RR cohort (12.68 ± 3.29% vs 10.81 ± 2.34%). Pacing with a single combined HF and RR population-optimised lead design achieves electrical resynchronization with near equivalence to personalised lead designs both in HF and RR anatomies. These findings suggest that although lead configurations have to be tailored to each patient, a single optimal lead design is sufficient to obtain near-optimal results across most patients. This study shows the potential of virtual clinical trials as tools to compare existing and explore new lead designs.

Seven challenges in the multiscale modeling of multicellular tissues

The growth and dynamics of multicellular tissues involve tightly regulated and coordinated morphogenetic cell behaviors, such as shape changes, movement, and division, which are governed by subcellular machinery and involve coupling through short- and long-range signals. A key challenge in the fields of developmental biology, tissue engineering and regenerative medicine is to understand how relationships between scales produce emergent tissue-scale behaviors. Recent advances in molecular biology, live-imaging and ex vivo techniques have revolutionized our ability to study these processes experimentally. To fully leverage these techniques and obtain a more comprehensive understanding of the causal relationships underlying tissue dynamics, computational modeling approaches are increasingly spanning multiple spatial and temporal scales, and are coupling cell shape, growth, mechanics, and signaling. Yet such models remain challenging: modeling at each scale requires different areas of technical skills, while integration across scales necessitates the solution to novel mathematical and computational problems. This review aims to summarize recent progress in multiscale modeling of multicellular tissues and to highlight ongoing challenges associated with the construction, implementation, interrogation, and validation of such models. This article is categorized under: Reproductive System Diseases > Computational Models Metabolic Diseases > Computational Models Cancer > Computational Models.

Computational modelling of the scoliotic spine: A literature review

Scoliosis is a deformity of the spine that in severe cases requires surgical treatment. There is still disagreement among clinicians as to what the aim of such treatment is as well as the optimal surgical technique. Numerical models can aid clinical decision-making by estimating the outcome of a given surgical intervention. This paper provided some background information on the modelling of the healthy spine and a review of the literature on scoliotic spine models, their validation, and their application. An overview of the methods and techniques used to construct scoliotic finite element and multibody models was given as well as the boundary conditions used in the simulations. The current limitations of the models were discussed as well as how such limitations are addressed in non-scoliotic spine models. Finally, future directions for the numerical modelling of scoliosis were addressed.

Collective Variational Inference for Personalized and Generative Physiological Modeling: A Case Study on Hemorrhage Resuscitation

OBJECTIVE

Individual physiological experiments typically provide useful but incomplete information about a studied physiological process. As a result, inferring the unknown parameters of a physiological model from experimental data is often challenging. The objective of this paper is to propose and illustrate the efficacy of a collective variational inference (C-VI) method, intended to reconcile low-information and heterogeneous data from a collection of experiments to produce robust personalized and generative physiological models.

METHODS

To derive the C-VI method, we utilize a probabilistic graphical model to impose structure on the available physiological data, and algorithmically characterize the graphical model using variational Bayesian inference techniques. To illustrate the efficacy of the C-VI method, we apply it to a case study on the mathematical modeling of hemorrhage resuscitation.

RESULTS

In the context of hemorrhage resuscitation modeling, the C-VI method could reconcile heterogeneous combinations of hematocrit, cardiac output, and blood pressure data across multiple experiments to obtain (i) robust personalized models along with associated measures of uncertainty and signal quality, and (ii) a generative model capable of reproducing the physiological behavior of the population.

CONCLUSION

The C-VI method facilitates the personalized and generative modeling of physiological processes in the presence of low-information and heterogeneous data.

SIGNIFICANCE

The resulting models provide a solid basis for the development and testing of interpretable physiological monitoring, decision-support, and closed-loop control algorithms.

Scientific and regulatory evaluation of mechanistic in silico drug and disease models in drug development: Building model credibility

The value of in silico methods in drug development and evaluation has been demonstrated repeatedly and convincingly. While their benefits are now unanimously recognized, international standards for their evaluation, accepted by all stakeholders involved, are still to be established. In this white paper, we propose a risk‐informed evaluation framework for mechanistic model credibility evaluation. To properly frame the proposed verification and validation activities, concepts such as context of use, regulatory impact and risk‐based analysis are discussed. To ensure common understanding between all stakeholders, an overview is provided of relevant in silico terminology used throughout this paper. To illustrate the feasibility of the proposed approach, we have applied it to three real case examples in the context of drug development, using a credibility matrix currently being tested as a quick‐start tool by regulators. Altogether, this white paper provides a practical approach to model evaluation, applicable in both scientific and regulatory evaluation contexts.

The first virtual patient-specific thrombectomy procedure

Treatment of acute ischemic stroke has been recently improved with the introduction of endovascular mechanical thrombectomy, a minimally invasive procedure able to remove a clot using aspiration devices and/or stent-retrievers. Despite the promising and encouraging results, improvements to the procedure and to the stent design are the focus of the recent efforts. Computational studies can pave the road to these improvements, providing their ability to describe and accurately reproduce a real procedure. A patient with ischemic stroke due to intracranial large vessel occlusion was selected and after the creation of the cerebral vasculature from computed tomography images and a histologic analysis to determine the clot composition, the entire thrombectomy procedure was virtually replicated. As in the real situation, the computational replica showed that two attempts were necessary to remove the clot, as a result of the position of the stent retriever with respect to the clot. Furthermore, the results indicated that clot fragmentation did not occur as the deformations were mainly in a compressive state without the possibility for clot cracks to propagate. The accurate representation of the procedure can be used as an important step for operative optimization planning and future improvements of stent designs.

Verification of an agent-based disease model of human Mycobacterium tuberculosis infection

Agent-based models (ABMs) are a powerful class of computational models widely used to simulate complex phenomena in many different application areas. However, one of the most critical aspects, poorly investigated in the literature, regards an important step of the model credibility assessment: solution verification. This study overcomes this limitation by proposing a general verification framework for ABMs that aims at evaluating the numerical errors associated with the model. A step-by-step procedure, which consists of two main verification studies (deterministic and stochastic model verification), is described in detail and applied to a specific mission critical scenario: the quantification of the numerical approximation error for UISS-TB, an ABM of the human immune system developed to predict the progression of pulmonary tuberculosis. Results provide indications on the possibility to use the proposed model verification workflow to systematically identify and quantify numerical approximation errors associated with UISS-TB and, in general, with any other ABMs.

In-silico trial of intracranial flow diverters replicates and expands insights from conventional clinical trials

The cost of clinical trials is ever-increasing. In-silico trials rely on virtual populations and interventions simulated using patient-specific models and may offer a solution to lower these costs. We present the flow diverter performance assessment (FD-PASS) in-silico trial, which models the treatment of intracranial aneurysms in 164 virtual patients with 82 distinct anatomies with a flow-diverting stent, using computational fluid dynamics to quantify post-treatment flow reduction. The predicted FD-PASS flow-diversion success rates replicate the values previously reported in three clinical trials. The in-silico approach allows broader investigation of factors associated with insufficient flow reduction than feasible in a conventional trial. Our findings demonstrate that in-silico trials of endovascular medical devices can: (i) replicate findings of conventional clinical trials, and (ii) perform virtual experiments and sub-group analyses that are difficult or impossible in conventional trials to discover new insights on treatment failure, e.g. in the presence of side-branches or hypertension.

In-silico trials rely on virtual populations and interventions simulated using patient-specific models and may offer a solution to lower costs. Here, the authors present the flow diverter performance assessment in-silico trial, which models the treatment of intracranial aneurysms with a flow-diverting stent.

From the real device to the digital twin: A coupled experimental-numerical strategy to investigate a novel bioresorbable vascular scaffold

The purpose of this work is to propose a workflow that couples experimental and computational activities aimed at developing a credible digital twin of a commercial coronary bioresorbable vascular scaffold when direct access to data about material mechanical properties is not possible. Such a situation is be faced when the manufacturer is not involved in the study, thus directly investigating the actual device is the only source of information available. The object of the work is the Fantom® Encore polymeric stent (REVA Medical) made of Tyrocore™. Four devices were purchased and used in mechanical tests that are easily reproducible in any mechanical laboratory, i.e. free expansion and uniaxial tension testing, the latter performed with protocols that emphasized the rate-dependent properties of the polymer. Given the complexity of the mechanical behaviour observed experimentally, it was chosen to use the Parallel Rehological Framework material model, already used in the literature to describe the behaviour of other polymers, such as PLLA. Calibration of the material model was based on simulations that replicate the tensile test performed on the device. Given the high number of material parameters, a plan of simulations was done to find the most suitable set, varying each parameter value in a feasible range and considering a single repetitive unit of the stent, neglecting residual stresses generated by crimping and expansion. This strategy resulted in a significant reduction of computational cost. The performance of the set of parameters thus identified was finally evaluated considering the whole delivery system, by comparing the experimental results with the data collected simulating free expansion and uniaxial tension testing. Moreover, radial force testing was numerically performed and compared with literature data. The obtained results demonstrated the effectiveness of the digital twin development pipeline, a path applicable to any commercial device whose geometric structure is based on repetitive units.

Reporting checklist for verification and validation of finite element analysis in orthopedic and trauma biomechanics

Finite element analysis (FEA) has become a fundamental tool for biomechanical investigations in the last decades. Despite several existing initiatives and guidelines for reporting on research methods and results, there are still numerous issues that arise when using computational models in biomechanical investigations. According to our knowledge, these problems and controversies lie mainly in the verification and validation (V&V) process as well as in the set-up and evaluation of FEA. This work aims to introduce a checklist including a report form defining recommendations for FEA in the field of Orthopedic and Trauma (O&T) biomechanics. Therefore, a checklist was elaborated which summarizes and explains the crucial methodologies for the V&V process. In addition, a report form has been developed which contains the most important steps for reporting future FEA. An example of the report form is shown, and a template is provided, which can be used as a uniform basis for future documentation. The future application of the presented report form will show whether serious errors in biomechanical investigations using FEA can be minimized by this checklist. Finally, the credibility of the FEA in the clinical area and the scientific exchange in the community regarding reproducibility and exchangeability can be improved.

Mathematical model of volume kinetics and renal function after burn injury and resuscitation

This paper presents a mathematical model of blood volume kinetics and renal function in response to burn injury and resuscitation, which is applicable to the development and non-clinical testing of burn resuscitation protocols and algorithms. Prior mathematical models of burn injury and resuscitation are not ideally suited to such applications due to their limited credibility in predicting blood volume and urinary output observed in wide-ranging burn patients as well as in incorporating contemporary knowledge of burn pathophysiology. Our mathematical model consists of an established multi-compartmental model of blood volume kinetics, a hybrid mechanistic-phenomenological model of renal function, and novel lumped-parameter models of burn-induced perturbations in volume kinetics and renal function equipped with contemporary knowledge on burn-related physiology and pathophysiology. Using the dataset collected from 16 sheep, we showed that our mathematical model can be characterized with physiologically plausible parameter values to accurately predict blood volume kinetic and renal function responses to burn injury and resuscitation on an individual basis against a wide range of pathophysiological variability. Pending validation in humans, our mathematical model may serve as an effective basis for in-depth understanding of complex burn-induced volume kinetic and renal function responses as well as development and non-clinical testing of burn resuscitation protocols and algorithms.

The combination of artificial intelligence and systems biology for intelligent vaccine design

INTRODUCTION

A new body of evidence depicts the applications of artificial intelligence and systems biology in vaccine design and development. The combination of both approaches shall revolutionize healthcare, accelerating clinical trial processes and reducing the costs and time involved in drug research and development.

AREAS COVERED

This review explores the basics of artificial intelligence and systems biology approaches in the vaccine development pipeline. The topics include a detailed description of epitope prediction tools for designing epitope-based vaccines and agent-based models for immune system response prediction, along with a focus on their potentiality to facilitate clinical trial phases.

EXPERT OPINION

Artificial intelligence and systems biology offer the opportunity to avoid the inefficiencies and failures that arise in the classical vaccine development pipeline. One promising solution is the combination of both methodologies in a multiscale perspective through an accurate pipeline. We are entering an 'in silico era' in which scientific partnerships, including a more and more increasing creation of an 'ecosystem' of collaboration and multidisciplinary approach, are relevant for addressing the long and risky road of vaccine discovery and development. In this context, regulatory guidance should be developed to qualify the in silico trials as evidence for intelligent vaccine development.

In silico trials: Verification, validation and uncertainty quantification of predictive models used in the regulatory evaluation of biomedical products

•

Regulators now consider also evidences produced in silico.

•

We need accepted methods to evaluate the credibility of models.

•

In this paper we describe the use of the ASME V&V-40 technical standard.

•

We also discuss its application to various types of modelling methods.

Historically, the evidences of safety and efficacy that companies provide to regulatory agencies as support to the request for marketing authorization of a new medical product have been produced experimentally, either in vitro or in vivo. More recently, regulatory agencies started receiving and accepting evidences obtained in silico, i.e. through modelling and simulation. However, before any method (experimental or computational) can be acceptable for regulatory submission, the method itself must be considered “qualified” by the regulatory agency. This involves the assessment of the overall “credibility” that such a method has in providing specific evidence for a given regulatory procedure. In this paper, we describe a methodological framework for the credibility assessment of computational models built using mechanistic knowledge of physical and chemical phenomena, in addition to available biological and physiological knowledge; these are sometimes referred to as “biophysical” models. Using guiding examples, we explore the definition of the context of use, the risk analysis for the definition of the acceptability thresholds, and the various steps of a comprehensive verification, validation and uncertainty quantification process, to conclude with considerations on the credibility of a prediction for a specific context of use. While this paper does not provide a guideline for the formal qualification process, which only the regulatory agencies can provide, we expect it to help researchers to better appreciate the extent of scrutiny required, which should be considered early on in the development/use of any (new) in silico evidence.