Active learning using adaptable task-based prioritisation

Supervised machine learning-based medical image computing applications necessitate expert label curation, while unlabelled image data might be relatively abundant. Active learning methods aim to prioritise a subset of available image data for expert annotation, for label-efficient model training. We develop a controller neural network that measures priority of images in a sequence of batches, as in batch-mode active learning, for multi-class segmentation tasks. The controller is optimised by rewarding positive task-specific performance gain, within a Markov decision process (MDP) environment that also optimises the task predictor. In this work, the task predictor is a segmentation network. A meta-reinforcement learning algorithm is proposed with multiple MDPs, such that the pre-trained controller can be adapted to a new MDP that contains data from different institutes and/or requires segmentation of different organs or structures within the abdomen. We present experimental results using multiple CT datasets from more than one thousand patients, with segmentation tasks of nine different abdominal organs, to demonstrate the efficacy of the learnt prioritisation controller function and its cross-institute and cross-organ adaptability. We show that the proposed adaptable prioritisation metric yields converging segmentation accuracy for a new kidney segmentation task, unseen in training, using between approximately 40% to 60% of labels otherwise required with other heuristic or random prioritisation metrics. For clinical datasets of limited size, the proposed adaptable prioritisation offers a performance improvement of 22.6% and 10.2% in Dice score, for tasks of kidney and liver vessel segmentation, respectively, compared to random prioritisation and alternative active sampling strategies.

PitSurgRT: real-time localization of critical anatomical structures in endoscopic pituitary surgery

PURPOSE

Endoscopic pituitary surgery entails navigating through the nasal cavity and sphenoid sinus to access the sella using an endoscope. This procedure is intricate due to the proximity of crucial anatomical structures (e.g. carotid arteries and optic nerves) to pituitary tumours, and any unintended damage can lead to severe complications including blindness and death. Intraoperative guidance during this surgery could support improved localization of the critical structures leading to reducing the risk of complications.

METHODS

A deep learning network PitSurgRT is proposed for real-time localization of critical structures in endoscopic pituitary surgery. The network uses high-resolution net (HRNet) as a backbone with a multi-head for jointly localizing critical anatomical structures while segmenting larger structures simultaneously. Moreover, the trained model is optimized and accelerated by using TensorRT. Finally, the model predictions are shown to neurosurgeons, to test their guidance capabilities.

RESULTS

Compared with the state-of-the-art method, our model significantly reduces the mean error in landmark detection of the critical structures from 138.76 to 54.40 pixels in a 1280 × 720-pixel image. Furthermore, the semantic segmentation of the most critical structure, sella, is improved by 4.39% IoU. The inference speed of the accelerated model achieves 298 frames per second with floating-point-16 precision. In the study of 15 neurosurgeons, 88.67% of predictions are considered accurate enough for real-time guidance.

CONCLUSION

The results from the quantitative evaluation, real-time acceleration, and neurosurgeon study demonstrate the proposed method is highly promising in providing real-time intraoperative guidance of the critical anatomical structures in endoscopic pituitary surgery.

The distinct roles of reinforcement learning between pre-procedure and intra-procedure planning for prostate biopsy

PURPOSE

Magnetic resonance (MR) imaging targeted prostate cancer (PCa) biopsy enables precise sampling of MR-detected lesions, establishing its importance in recommended clinical practice. Planning for the ultrasound-guided procedure involves pre-selecting needle sampling positions. However, performing this procedure is subject to a number of factors, including MR-to-ultrasound registration, intra-procedure patient movement and soft tissue motions. When a fixed pre-procedure planning is carried out without intra-procedure adaptation, these factors will lead to sampling errors which could cause false positives and false negatives. Reinforcement learning (RL) has been proposed for procedure plannings on similar applications such as this one, because intelligent agents can be trained for both pre-procedure and intra-procedure planning. However, it is not clear if RL is beneficial when it comes to addressing these intra-procedure errors.

METHODS

In this work, we develop and compare imitation learning (IL), supervised by demonstrations of predefined sampling strategy, and RL approaches, under varying degrees of intra-procedure motion and registration error, to represent sources of targeting errors likely to occur in an intra-operative procedure.

RESULTS

Based on results using imaging data from 567 PCa patients, we demonstrate the efficacy and value in adopting RL algorithms to provide intelligent intra-procedure action suggestions, compared to IL-based planning supervised by commonly adopted policies.

CONCLUSIONS

The improvement in biopsy sampling performance for intra-procedure planning has not been observed in experiments with only pre-procedure planning. These findings suggest a strong role for RL in future prospective studies which adopt intra-procedure planning. Our open source code implementation is available here .

The value of Augmented Reality in surgery - A usability study on laparoscopic liver surgery

Augmented Reality (AR) is considered to be a promising technology for the guidance of laparoscopic liver surgery. By overlaying pre-operative 3D information of the liver and internal blood vessels on the laparoscopic view, surgeons can better understand the location of critical structures. In an effort to enable AR, several authors have focused on the development of methods to obtain an accurate alignment between the laparoscopic video image and the pre-operative 3D data of the liver, without assessing the benefit that the resulting overlay can provide during surgery. In this paper, we present a study that aims to assess quantitatively and qualitatively the value of an AR overlay in laparoscopic surgery during a simulated surgical task on a phantom setup. We design a study where participants are asked to physically localise pre-operative tumours in a liver phantom using three image guidance conditions - a baseline condition without any image guidance, a condition where the 3D surfaces of the liver are aligned to the video and displayed on a black background, and a condition where video see-through AR is displayed on the laparoscopic video. Using data collected from a cohort of 24 participants which include 12 surgeons, we observe that compared to the baseline, AR decreases the median localisation error of surgeons on non-peripheral targets from 25.8 mm to 9.2 mm. Using subjective feedback, we also identify that AR introduces usability improvements in the surgical task and increases the perceived confidence of the users. Between the two tested displays, the majority of participants preferred to use the AR overlay instead of navigated view of the 3D surfaces on a separate screen. We conclude that AR has the potential to improve performance and decision making in laparoscopic surgery, and that improvements in overlay alignment accuracy and depth perception should be pursued in the future.

Prototypical few-shot segmentation for cross-institution male pelvic structures with spatial registration

The prowess that makes few-shot learning desirable in medical image analysis is the efficient use of the support image data, which are labelled to classify or segment new classes, a task that otherwise requires substantially more training images and expert annotations. This work describes a fully 3D prototypical few-shot segmentation algorithm, such that the trained networks can be effectively adapted to clinically interesting structures that are absent in training, using only a few labelled images from a different institute. First, to compensate for the widely recognised spatial variability between institutions in episodic adaptation of novel classes, a novel spatial registration mechanism is integrated into prototypical learning, consisting of a segmentation head and an spatial alignment module. Second, to assist the training with observed imperfect alignment, support mask conditioning module is proposed to further utilise the annotation available from the support images. Extensive experiments are presented in an application of segmenting eight anatomical structures important for interventional planning, using a data set of 589 pelvic T2-weighted MR images, acquired at seven institutes. The results demonstrate the efficacy in each of the 3D formulation, the spatial registration, and the support mask conditioning, all of which made positive contributions independently or collectively. Compared with the previously proposed 2D alternatives, the few-shot segmentation performance was improved with statistical significance, regardless whether the support data come from the same or different institutes.

Long-term Dependency for 3D Reconstruction of Freehand Ultrasound Without External Tracker

OBJECTIVE

Reconstructing freehand ultrasound in 3D without any external tracker has been a long-standing challenge in ultrasound-assisted procedures. We aim to define new ways of parameterising long-term dependencies, and evaluate the performance.

METHODS

First, long-term dependency is encoded by transformation positions within a frame sequence. This is achieved by combining a sequence model with a multi-transformation prediction. Second, two dependency factors are proposed, anatomical image content and scanning protocol, for contributing towards accurate reconstruction. Each factor is quantified experimentally by reducing respective training variances.

RESULTS

1) The added long-term dependency up to 400 frames at 20 frames per second (fps) indeed improved reconstruction, with an up to 82.4% lowered accumulated error, compared with the baseline performance. The improvement was found to be dependent on sequence length, transformation interval and scanning protocol and, unexpectedly, not on the use of recurrent networks with long-short term modules; 2) Decreasing either anatomical or protocol variance in training led to poorer reconstruction accuracy. Interestingly, greater performance was gained from representative protocol patterns, than from representative anatomical features.

CONCLUSION

The proposed algorithm uses hyperparameter tuning to effectively utilise long-term dependency. The proposed dependency factors are of practical significance in collecting diverse training data, regulating scanning protocols and developing efficient networks.

SIGNIFICANCE

The proposed new methodology with publicly available volunteer data and code for parametersing the long-term dependency, experimentally shown to be valid sources of performance improvement, which could potentially lead to better model development and practical optimisation of the reconstruction application.

Image-guidance in endoscopic pituitary surgery: an in-silico study of errors involved in tracker-based techniques

Background

Endoscopic endonasal surgery is an established minimally invasive technique for resecting pituitary adenomas. However, understanding orientation and identifying critical neurovascular structures in this anatomically dense region can be challenging. In clinical practice, commercial navigation systems use a tracked pointer for guidance. Augmented Reality (AR) is an emerging technology used for surgical guidance. It can be tracker based or vision based, but neither is widely used in pituitary surgery.

Methods

This pre-clinical study aims to assess the accuracy of tracker-based navigation systems, including those that allow for AR. Two setups were used to conduct simulations: (1) the standard pointer setup, tracked by an infrared camera; and (2) the endoscope setup that allows for AR, using reflective markers on the end of the endoscope, tracked by infrared cameras. The error sources were estimated by calculating the Euclidean distance between a point's true location and the point's location after passing it through the noisy system. A phantom study was then conducted to verify the in-silico simulation results and show a working example of image-based navigation errors in current methodologies.

Results

The errors of the tracked pointer and tracked endoscope simulations were 1.7 and 2.5 mm respectively. The phantom study showed errors of 2.14 and 3.21 mm for the tracked pointer and tracked endoscope setups respectively.

Discussion

In pituitary surgery, precise neighboring structure identification is crucial for success. However, our simulations reveal that the errors of tracked approaches were too large to meet the fine error margins required for pituitary surgery. In order to achieve the required accuracy, we would need much more accurate tracking, better calibration and improved registration techniques.

3D Generative Model Latent Disentanglement via Local Eigenprojection

Designing realistic digital humans is extremely complex. Most data-driven generative models used to simplify the creation of their underlying geometric shape do not offer control over the generation of local shape attributes. In this paper, we overcome this limitation by introducing a novel loss function grounded in spectral geometry and applicable to different neural-network-based generative models of 3D head and body meshes. Encouraging the latent variables of mesh variational autoencoders (VAEs) or generative adversarial networks (GANs) to follow the local eigenprojections of identity attributes, we improve latent disentanglement and properly decouple the attribute creation. Experimental results show that our local eigenprojection disentangled (LED) models not only offer improved disentanglement with respect to the state-of-the-art, but also maintain good generation capabilities with training times comparable to the vanilla implementations of the models. Our code and pre-trained models are available at github.com/simofoti/LocalEigenprojDisentangled.

Evaluation of a calibration rig for stereo laparoscopes

BACKGROUND

Accurate camera and hand-eye calibration are essential to ensure high-quality results in image-guided surgery applications. The process must also be able to be undertaken by a nonexpert user in a surgical setting.

PURPOSE

This work seeks to identify a suitable method for tracked stereo laparoscope calibration within theater.

METHODS

A custom calibration rig, to enable rapid calibration in a surgical setting, was designed. The rig was compared against freehand calibration. Stereo reprojection, stereo reconstruction, tracked stereo reprojection, and tracked stereo reconstruction error metrics were used to evaluate calibration quality.

RESULTS

Use of the calibration rig reduced mean errors: reprojection (1.47 mm [SD 0.13] vs. 3.14 mm [SD 2.11], p-value 1e-8), reconstruction (1.37 px [SD 0.10] vs. 10.10 px [SD 4.54], p-value 6e-7), and tracked reconstruction (1.38 mm [SD 0.10] vs. 12.64 mm [SD 4.34], p-value 1e-6) compared with freehand calibration. The use of a ChArUco pattern yielded slightly lower reprojection errors, while a dot grid produced lower reconstruction errors and was more robust under strong global illumination.

CONCLUSION

The use of the calibration rig results in a statistically significant decrease in calibration error metrics, versus freehand calibration, and represents the preferred approach for use in the operating theater.

Enhanced Surgeons: Understanding the Design of Augmented Reality Instructions for Keyhole Surgery

It is important to understand how to design AR content for surgical contexts to mitigate the risk of distracting the surgeons. In this work, we test information overlays for AR guidance during keyhole surgery. We performed a preliminary evaluation of a prototype, focusing on the effects of colour, opacity, and information representation. Our work contributes insights into the design of AR guidance in surgery settings and a foundation for future research on visualisation design for surgical AR.

The Effect of Luminance on Depth Perception in Augmented Reality Guided Laparoscopic Surgery

Depth perception is a major issue in surgical augmented reality (AR) with limited research conducted in this scientific area. This study establishes a relationship between luminance and depth perception. This can be used to improve visualisation design for AR overlay in laparoscopic surgery, providing surgeons a more accurate perception of the anatomy intraoperatively. Two experiments were conducted to determine this relationship. First, an online study with 59 participants from the general public, and second, an in-person study with 10 surgeons as participants. We developed 2 open-source software tools utilising SciKit-Surgery libraries to enable these studies and any future research. Our findings demonstrate that the higher the relative luminance, the closer a structure is perceived to the operating camera. Furthermore, the higher the luminance contrast between the two structures, the higher the depth distance perceived. The quantitative results from both experiments are in agreement, indicating that online recruitment of the general public can be helpful in similar studies. An observation made by the surgeons from the in-person study was that the light source used in laparoscopic surgery plays a role in depth perception. This is due to its varying positioning and brightness which could affect the perception of the overlaid AR. We found that luminance directly correlates with depth perception for both surgeons and the general public, regardless of other depth cues. Future research may focus on comparing different colours used in surgical AR and using a mock operating room (OR) with varying light sources and positions.

Assessing safety and treatment efficacy of running on intervertebral discs (ASTEROID) in adults with chronic low back pain: protocol for a randomised controlled trial

Poor intervertebral disc (IVD) health is associated with low back pain (LBP). This 12-week parallel randomised controlled trial will evaluate the efficacy of a progressive interval running programme on IVD health and other clinical outcomes in adults with chronic LBP. Participants will be randomised to either a digitally delivered progressive interval running programme or waitlist control. Participants randomised to the running programme will receive three individually tailored 30 min community-based sessions per week over 12 weeks. The waitlist control will undergo no formal intervention. All participants will be assessed at baseline, 6 and 12 weeks. Primary outcomes are IVD health (lumbar IVD T2 via MRI), average LBP intensity over the prior week (100-point visual analogue scale) and disability (Oswestry Disability Index). Secondary outcomes include a range of clinical measures. All outcomes will be analysed using linear mixed models. This study has received ethical approval from the Deakin University Human Research Ethics Committee (ID: 2022-162). All participants will provide informed written consent before participation. Regardless of the results, the findings of this study will be disseminated, and anonymised data will be shared via an online repository. This will be the first study to evaluate whether a progressive interval running programme can improve IVD health in adults with chronic LBP. Identifying conservative options to improve IVD health in this susceptible population group has the potential to markedly reduce the burden of disease. This study was registered via the Australian New Zealand Clinical Trials Registry on 29 September 2022 (ACTRN12622001276741).

The Effectiveness of Exercise Physiology Services During the COVID-19 Pandemic: A Pragmatic Cohort Study

BACKGROUND

The COVID-19 pandemic markedly changed how healthcare services are delivered and telehealth delivery has increased worldwide. Whether changes in healthcare delivery borne from the COVID-19 pandemic impact effectiveness is unknown. Therefore, we examined the effectiveness of exercise physiology services provided during the COVID-19 pandemic.

METHODS

This prospective cohort study included 138 clients who received exercise physiology services during the initial COVID-19 pandemic. Outcome measures of interest were EQ-5D-5L, EQ-VAS, patient-specific functional scale, numeric pain rating scale and goal attainment scaling.

RESULTS

Most (59%, n = 82) clients received in-person delivery only, whereas 8% (n = 11) received telehealth delivery only and 33% (n = 45) received a combination of delivery modes. Mean (SD) treatment duration was 11 (7) weeks and included 12 (6) sessions lasting 48 (9) minutes. The majority (73%, n = 101) of clients completed > 80% of exercise sessions. Exercise physiology improved mobility by 14% (β = 0.23, P = 0.003), capacity to complete usual activities by 18% (β = 0.29, P < 0.001), capacity to complete important activities that the client was unable to do or having difficulty performing by 54% (β = 2.46, P < 0.001), current pain intensity by 16% (β = - 0.55, P = 0.038) and goal attainment scaling t-scores by 50% (β = 18.37, P < 0.001). Effectiveness did not differ between delivery modes (all: P > 0.087).

CONCLUSIONS

Exercise physiology services provided during the COVID-19 pandemic improved a range of client-reported outcomes regardless of delivery mode. Further exploration of cost-effectiveness is warranted.

CAL-Tutor: A HoloLens 2 Application for Training in Obstetric Sonography and User Motion Data Recording

Obstetric ultrasound (US) training teaches the relationship between foetal anatomy and the viewed US slice to enable navigation to standardised anatomical planes (head, abdomen and femur) where diagnostic measurements are taken. This process is difficult to learn, and results in considerable inter-operator variability. We propose the CAL-Tutor system for US training based on a US scanner and phantom, where a model of both the baby and the US slice are displayed to the trainee in its physical location using the HoloLens 2. The intention is that AR guidance will shorten the learning curve for US trainees and improve spatial awareness. In addition to the AR guidance, we also record many data streams to assess user motion and the learning process. The HoloLens 2 provides eye gaze, head and hand position, ARToolkit and NDI Aurora tracking gives the US probe positions and an external camera records the overall scene. These data can provide a rich source for further analysis, such as distinguishing expert from novice motion. We have demonstrated the system in a sample of engineers. Feedback suggests that the system helps novice users navigate the US probe to the standard plane. The data capture is successful and initial data visualisations show that meaningful information about user behaviour can be captured. Initial feedback is encouraging and shows improved user assessment where AR guidance is provided.

Cross-Modality Image Registration Using a Training-Time Privileged Third Modality

In this work, we consider the task of pairwise cross-modality image registration, which may benefit from exploiting additional images available only at training time from an additional modality that is different to those being registered. As an example, we focus on aligning intra-subject multiparametric Magnetic Resonance (mpMR) images, between T2-weighted (T2w) scans and diffusion-weighted scans with high b-value (DWI [Formula: see text]). For the application of localising tumours in mpMR images, diffusion scans with zero b-value (DWI [Formula: see text]) are considered easier to register to T2w due to the availability of corresponding features. We propose a learning from privileged modality algorithm, using a training-only imaging modality DWI [Formula: see text], to support the challenging multi-modality registration problems. We present experimental results based on 369 sets of 3D multiparametric MRI images from 356 prostate cancer patients and report, with statistical significance, a lowered median target registration error of 4.34 mm, when registering the holdout DWI [Formula: see text] and T2w image pairs, compared with that of 7.96 mm before registration. Results also show that the proposed learning-based registration networks enabled efficient registration with comparable or better accuracy, compared with a classical iterative algorithm and other tested learning-based methods with/without the additional modality. These compared algorithms also failed to produce any significantly improved alignment between DWI [Formula: see text] and T2w in this challenging application.

Software tool for visualization of a probabilistic map of the epileptogenic zone from seizure semiologies

Around one third of epilepsies are drug-resistant. For these patients, seizures may be reduced or cured by surgically removing the epileptogenic zone (EZ), which is the portion of the brain giving rise to seizures. If noninvasive data are not sufficiently lateralizing or localizing, the EZ may need to be localized by precise implantation of intracranial electroencephalography (iEEG) electrodes. The choice of iEEG targets is influenced by clinicians' experience and personal knowledge of the literature, which leads to substantial variations in implantation strategies across different epilepsy centers. The clinical diagnostic pathway for surgical planning could be supported and standardized by an objective tool to suggest EZ locations, based on the outcomes of retrospective clinical cases reported in the literature. We present an open-source software tool that presents clinicians with an intuitive and data-driven visualization to infer the location of the symptomatogenic zone, that may overlap with the EZ. The likely EZ is represented as a probabilistic map overlaid on the patient's images, given a list of seizure semiologies observed in that specific patient. We demonstrate a case study on retrospective data from a patient treated in our unit, who underwent resective epilepsy surgery and achieved 1-year seizure freedom after surgery. The resected brain structures identified as EZ location overlapped with the regions highlighted by our tool, demonstrating its potential utility.

Phase 0 Study of Vandetanib-Eluting Radiopaque Embolics as a Preoperative Embolization Treatment in Patients with Resectable Liver Malignancies

PURPOSE

To assess the safety and tolerability of a vandetanib-eluting radiopaque embolic (BTG-002814) for transarterial chemoembolization (TACE) in patients with resectable liver malignancies.

MATERIALS AND METHODS

The VEROnA clinical trial was a first-in-human, phase 0, single-arm, window-of-opportunity study. Eligible patients were aged ≥18 years and had resectable hepatocellular carcinoma (HCC) (Child-Pugh A) or metastatic colorectal cancer (mCRC). Patients received 1 mL of BTG-002814 transarterially (containing 100 mg of vandetanib) 7-21 days prior to surgery. The primary objectives were to establish the safety and tolerability of BTG-002814 and determine the concentrations of vandetanib and the N-desmethyl vandetanib metabolite in the plasma and resected liver after treatment. Biomarker studies included circulating proangiogenic factors, perfusion computed tomography, and dynamic contrast-enhanced magnetic resonance imaging.

RESULTS

Eight patients were enrolled: 2 with HCC and 6 with mCRC. There was 1 grade 3 adverse event (AE) before surgery and 18 after surgery; 6 AEs were deemed to be related to BTG-002814. Surgical resection was not delayed. Vandetanib was present in the plasma of all patients 12 days after treatment, with a mean maximum concentration of 24.3 ng/mL (standard deviation ± 13.94 ng/mL), and in resected liver tissue up to 32 days after treatment (441-404,000 ng/g). The median percentage of tumor necrosis was 92.5% (range, 5%-100%). There were no significant changes in perfusion imaging parameters after TACE.

CONCLUSIONS

BTG-002814 has an acceptable safety profile in patients before surgery. The presence of vandetanib in the tumor specimens up to 32 days after treatment suggests sustained anticancer activity, while the low vandetanib levels in the plasma suggest minimal release into the systemic circulation. Further evaluation of this TACE combination is warranted in dose-finding and efficacy studies.

Gesture Recognition in Robotic Surgery With Multimodal Attention

Automatically recognising surgical gestures from surgical data is an important building block of automated activity recognition and analytics, technical skill assessment, intra-operative assistance and eventually robotic automation. The complexity of articulated instrument trajectories and the inherent variability due to surgical style and patient anatomy make analysis and fine-grained segmentation of surgical motion patterns from robot kinematics alone very difficult. Surgical video provides crucial information from the surgical site with context for the kinematic data and the interaction between the instruments and tissue. Yet sensor fusion between the robot data and surgical video stream is non-trivial because the data have different frequency, dimensions and discriminative capability. In this paper, we integrate multimodal attention mechanisms in a two-stream temporal convolutional network to compute relevance scores and weight kinematic and visual feature representations dynamically in time, aiming to aid multimodal network training and achieve effective sensor fusion. We report the results of our system on the JIGSAWS benchmark dataset and on a new in vivo dataset of suturing segments from robotic prostatectomy procedures. Our results are promising and obtain multimodal prediction sequences with higher accuracy and better temporal structure than corresponding unimodal solutions. Visualization of attention scores also gives physically interpretable insights on network understanding of strengths and weaknesses of each sensor.

Voice-Assisted Image Labeling for Endoscopic Ultrasound Classification Using Neural Networks

Ultrasound imaging is a commonly used technology for visualising patient anatomy in real-time during diagnostic and therapeutic procedures. High operator dependency and low reproducibility make ultrasound imaging and interpretation challenging with a steep learning curve. Automatic image classification using deep learning has the potential to overcome some of these challenges by supporting ultrasound training in novices, as well as aiding ultrasound image interpretation in patient with complex pathology for more experienced practitioners. However, the use of deep learning methods requires a large amount of data in order to provide accurate results. Labelling large ultrasound datasets is a challenging task because labels are retrospectively assigned to 2D images without the 3D spatial context available in vivo or that would be inferred while visually tracking structures between frames during the procedure. In this work, we propose a multi-modal convolutional neural network (CNN) architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. We use a CNN composed of two branches, one for voice data and another for image data, which are joined to predict image labels from the spoken names of anatomical landmarks. The network was trained using recorded verbal comments from expert operators. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels. We conclude that the addition of spoken commentaries can increase the performance of ultrasound image classification, and eliminate the burden of manually labelling large EUS datasets necessary for deep learning applications.

SciKit-SurgeryGlenoid, an Open Source Toolkit for Glenoid Version Measurement

Correct understanding of the geometry of the glenoid (the socket of the shoulder joint) is key to successful planning of shoulder replacement surgery. This surgery typically involves placing an implant in the shoulder joint to restore joint function. The most relevant geometry is the glenoid version, which is the angular orientation of the glenoid surface relative to the long axis of the scapula in the axial plane. However, measuring the glenoid version is not straightforward and there are multiple measurement methods in the literature and used in commercial planning software. In this paper we introduce SciKit-SurgeryGlenoid, an open source toolkit for the measurement of glenoid version. SciKit-SurgeryGlenoid contains implementations of the 4 most frequently used glenoid version measurement algorithms enabling easy and unbiased comparison of the different techniques. We present the results of using the software on 10 sets of pre-operative CT scans taken from patients who have subsequently undergone shoulder replacement surgery. We further compare these results with those obtained from a commercial implant planning software. SciKit-SurgeryGlenoid currently requires manual segmentation of the relevant anatomical features for each method. Future work will look at automating the segmentation process to build an automatic and repeatable pipeline from CT or radiograph to quantitative glenoid version measurement.

Deep hashing for global registration of untracked 2D laparoscopic ultrasound to CT

PURPOSE

The registration of Laparoscopic Ultrasound (LUS) to CT can enhance the safety of laparoscopic liver surgery by providing the surgeon with awareness on the relative positioning between critical vessels and a tumour. In an effort to provide a translatable solution for this poorly constrained problem, Content-based Image Retrieval (CBIR) based on vessel information has been suggested as a method for obtaining a global coarse registration without using tracking information. However, the performance of these frameworks is limited by the use of non-generalisable handcrafted vessel features.

METHODS

We propose the use of a Deep Hashing (DH) network to directly convert vessel images from both LUS and CT into fixed size hash codes. During training, these codes are learnt from a patient-specific CT scan by supplying the network with triplets of vessel images which include both a registered and a mis-registered pair. Once hash codes have been learnt, they can be used to perform registration with CBIR methods.

RESULTS

We test a CBIR pipeline on 11 sequences of untracked LUS distributed across 5 clinical cases. Compared to a handcrafted feature approach, our model improves the registration success rate significantly from 48% to 61%, considering a 20 mm error as the threshold for a successful coarse registration.

CONCLUSIONS

We present the first DH framework for interventional multi-modal registration tasks. The presented approach is easily generalisable to other registration problems, does not require annotated data for training, and may promote the translation of these techniques.

Large scale simulation of labeled intraoperative scenes in unity

PURPOSE

The use of synthetic or simulated data has the potential to greatly improve the availability and volume of training data for image guided surgery and other medical applications, where access to real-life training data is limited.

METHODS

By using the Unity game engine, complex intraoperative scenes can be simulated. The Unity Perception package allows for randomisation of paremeters within the scene, and automatic labelling, to make simulating large data sets a trivial operation. In this work, the approach has been prototyped for liver segmentation from laparoscopic video images. 50,000 simulated images were used to train a U-Net, without the need for any manual labelling. The use of simulated data was compared against a model trained with 950 manually labelled laparoscopic images.

RESULTS

When evaluated on data from 10 separate patients, synthetic data outperformed real data in 4 out of 10 cases. Average DICE scores across the 10 cases were 0.59 (synthetic data), 0.64 (real data) and 0.75 (both synthetic and real data).

CONCLUSION

Synthetic data generated using this method is able to make valid inferences on real data, with average performance slightly below models trained on real data. The use of the simulated data for pre-training boosts model performance, when compared with training on real data only.

Probabilistic landscape of seizure semiology localizing values

Semiology describes the evolution of symptoms and signs during epileptic seizures and contributes to the evaluation of individuals with focal drug-resistant epilepsy for curative resection. Semiology varies in complexity from elementary sensorimotor seizures arising from primary cortex to complex behaviours and automatisms emerging from distributed cerebral networks. Detailed semiology interpreted by expert epileptologists may point towards the likely site of seizure onset, but this process is subjective. No study has captured the variances in semiological localizing values in a data-driven manner to allow objective and probabilistic determinations of implicated networks and nodes. We curated an open data set from the epilepsy literature, in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, linking semiology to hierarchical brain localizations. A total of 11 230 data points were collected from 4643 patients across 309 articles, labelled using ground truths (postoperative seizure-freedom, concordance of imaging and neurophysiology, and/or invasive EEG) and a designation method that distinguished between semiologies arising from a predefined cortical region and descriptions of neuroanatomical localizations responsible for generating a particular semiology. This allowed us to mitigate temporal lobe publication bias by filtering studies that preselected patients based on prior knowledge of their seizure foci. Using this data set, we describe the probabilistic landscape of semiological localizing values as forest plots at the resolution of seven major brain regions: temporal, frontal, cingulate, parietal, occipital, insula, and hypothalamus, and five temporal subregions. We evaluated the intrinsic value of any one semiology over all other ictal manifestations. For example, epigastric auras implicated the temporal lobe with 83% probability when not accounting for the publication bias that favoured temporal lobe epilepsies. Unbiased results for a prior distribution of cortical localizations revised the prevalence of temporal lobe epilepsies from 66% to 44%. Therefore, knowledge about the presence of epigastric auras updates localization to the temporal lobe with an odds ratio (OR) of 2.4 [CI_95% (1.9, 2.9); and specifically, mesial temporal structures OR: 2.8 (2.3, 2.9)], attesting the value of epigastric auras. As a further example, although head version is thought to implicate the frontal lobes, it did not add localizing value compared with the prior distribution of cortical localizations [OR: 0.9 (0.7, 1.2)]. Objectification of the localizing values of the 12 most common semiologies provides a complementary view of brain dysfunction to that of lesion-deficit mappings, as instead of linking brain regions to phenotypic-deficits, semiological phenotypes are linked back to brain sources. This work enables coupling of seizure propagation with ictal manifestations, and clinical support algorithms for localizing seizure phenotypes.

Radiopaque drug-eluting embolisation beads as fiducial markers for stereotactic liver radiotherapy

Objective:

To determine the feasibility of using radiopaque (RO) beads as direct tumour surrogates for image-guided radiotherapy (IGRT) in patients with liver tumours after transarterial chemoembolisation (TACE).

Methods:

A novel vandetanib-eluting RO bead was delivered via TACE as part of a first-in-human clinical trial in patients with either hepatocellular carcinoma or liver metastases from colorectal cancer. Following TACE, patients underwent simulated radiotherapy imaging with four-dimensional computed tomography (4D-CT) and cone-beam CT (CBCT) imaging. RO beads were contoured using automated thresholding, and feasibility of matching between the simulated radiotherapy planning dataset (AVE-IP image from 4D data) and CBCT scans assessed. Additional kV, MV, helical CT and CBCT images of RO beads were obtained using an in-house phantom. Stability of RO bead position was assessed by comparing 4D-CT imaging to CT scans taken 6–20 days following TACE.

Results:

Eight patients were treated and 4D-CT and CBCT images acquired. RO beads were visible on 4D-CT and CBCT images in all cases and matching successfully performed. Differences in centre of mass of RO beads between CBCT and simulated radiotherapy planning scans (AVE-IP dataset) were 2.0 mm mediolaterally, 1.7 mm anteroposteriorally and 3.5 mm craniocaudally. RO beads in the phantom were visible on all imaging modalities assessed. RO bead position remained stable up to 29 days post TACE.

Conclusion:

RO beads are visible on IGRT imaging modalities, showing minimal artefact. They can be used for on-set matching with CBCT and remain stable over time.

Advances in knowledge:

The role of RO beads as fiducial markers for stereotactic liver radiotherapy is feasible and warrants further exploration as a combination therapy approach.

Zero-Shot Super-Resolution With a Physically-Motivated Downsampling Kernel for Endomicroscopy

Super-resolution (SR) methods have seen significant advances thanks to the development of convolutional neural networks (CNNs). CNNs have been successfully employed to improve the quality of endomicroscopy imaging. Yet, the inherent limitation of research on SR in endomicroscopy remains the lack of ground truth high-resolution (HR) images, commonly used for both supervised training and reference-based image quality assessment (IQA). Therefore, alternative methods, such as unsupervised SR are being explored. To address the need for non-reference image quality improvement, we designed a novel zero-shot super-resolution (ZSSR) approach that relies only on the endomicroscopy data to be processed in a self-supervised manner without the need for ground-truth HR images. We tailored the proposed pipeline to the idiosyncrasies of endomicroscopy by introducing both: a physically-motivated Voronoi downscaling kernel accounting for the endomicroscope's irregular fibre-based sampling pattern, and realistic noise patterns. We also took advantage of video sequences to exploit a sequence of images for self-supervised zero-shot image quality improvement. We run ablation studies to assess our contribution in regards to the downscaling kernel and noise simulation. We validate our methodology on both synthetic and original data. Synthetic experiments were assessed with reference-based IQA, while our results for original images were evaluated in a user study conducted with both expert and non-expert observers. The results demonstrated superior performance in image quality of ZSSR reconstructions in comparison to the baseline method. The ZSSR is also competitive when compared to supervised single-image SR, especially being the preferred reconstruction technique by experts.

Vessel segmentation for automatic registration of untracked laparoscopic ultrasound to CT of the liver

Purpose:

Registration of Laparoscopic Ultrasound (LUS) to a pre-operative scan such as Computed Tomography (CT) using blood vessel information has been proposed as a method to enable image-guidance for laparoscopic liver resection. Currently, there are solutions for this problem that can potentially enable clinical translation by bypassing the need for a manual initialisation and tracking information. However, no reliable framework for the segmentation of vessels in 2D untracked LUS images has been presented.

Methods:

We propose the use of 2D UNet for the segmentation of liver vessels in 2D LUS images. We integrate these results in a previously developed registration method, and show the feasibility of a fully automatic initialisation to the LUS to CT registration problem without a tracking device.

Results:

We validate our segmentation using LUS data from 6 patients. We test multiple models by placing patient datasets into different combinations of training, testing and hold-out, and obtain mean Dice scores ranging from 0.543 to 0.706. Using these segmentations, we obtain registration accuracies between 6.3 and 16.6 mm in 50% of cases.

Conclusions:

We demonstrate the first instance of deep learning (DL) for the segmentation of liver vessels in LUS. Our results show the feasibility of UNet in detecting multiple vessel instances in 2D LUS images, and potentially automating a LUS to CT registration pipeline.

Gesture Recognition in Robotic Surgery: A Review

OBJECTIVE

Surgical activity recognition is a fundamental step in computer-assisted interventions. This paper reviews the state-of-the-art in methods for automatic recognition of fine-grained gestures in robotic surgery focusing on recent data-driven approaches and outlines the open questions and future research directions.

METHODS

An article search was performed on 5 bibliographic databases with the following search terms: robotic, robot-assisted, JIGSAWS, surgery, surgical, gesture, fine-grained, surgeme, action, trajectory, segmentation, recognition, parsing. Selected articles were classified based on the level of supervision required for training and divided into different groups representing major frameworks for time series analysis and data modelling.

RESULTS

A total of 52 articles were reviewed. The research field is showing rapid expansion, with the majority of articles published in the last 4 years. Deep-learning-based temporal models with discriminative feature extraction and multi-modal data integration have demonstrated promising results on small surgical datasets. Currently, unsupervised methods perform significantly less well than the supervised approaches.

CONCLUSION

The development of large and diverse open-source datasets of annotated demonstrations is essential for development and validation of robust solutions for surgical gesture recognition. While new strategies for discriminative feature extraction and knowledge transfer, or unsupervised and semi-supervised approaches, can mitigate the need for data and labels, they have not yet been demonstrated to achieve comparable performance. Important future research directions include detection and forecast of gesture-specific errors and anomalies.

SIGNIFICANCE

This paper is a comprehensive and structured analysis of surgical gesture recognition methods aiming to summarize the status of this rapidly evolving field.

Integrated multi-modality image-guided navigation for neurosurgery: open-source software platform using state-of-the-art clinical hardware

PURPOSE

Image-guided surgery (IGS) is an integral part of modern neuro-oncology surgery. Navigated ultrasound provides the surgeon with reconstructed views of ultrasound data, but no commercial system presently permits its integration with other essential non-imaging-based intraoperative monitoring modalities such as intraoperative neuromonitoring. Such a system would be particularly useful in skull base neurosurgery.

METHODS

We established functional and technical requirements of an integrated multi-modality IGS system tailored for skull base surgery with the ability to incorporate: (1) preoperative MRI data and associated 3D volume reconstructions, (2) real-time intraoperative neurophysiological data and (3) live reconstructed 3D ultrasound. We created an open-source software platform to integrate with readily available commercial hardware. We tested the accuracy of the system's ultrasound navigation and reconstruction using a polyvinyl alcohol phantom model and simulated the use of the complete navigation system in a clinical operating room using a patient-specific phantom model.

RESULTS

Experimental validation of the system's navigated ultrasound component demonstrated accuracy of [Formula: see text] and a frame rate of 25 frames per second. Clinical simulation confirmed that system assembly was straightforward, could be achieved in a clinically acceptable time of [Formula: see text] and performed with a clinically acceptable level of accuracy.

CONCLUSION

We present an integrated open-source research platform for multi-modality IGS. The present prototype system was tailored for neurosurgery and met all minimum design requirements focused on skull base surgery. Future work aims to optimise the system further by addressing the remaining target requirements.

Registration of Untracked 2D Laparoscopic Ultrasound to CT Images of the Liver Using Multi-Labelled Content-Based Image Retrieval

Laparoscopic Ultrasound (LUS) is recommended as a standard-of-care when performing laparoscopic liver resections as it images sub-surface structures such as tumours and major vessels. Given that LUS probes are difficult to handle and some tumours are iso-echoic, registration of LUS images to a pre-operative CT has been proposed as an image-guidance method. This registration problem is particularly challenging due to the small field of view of LUS, and usually depends on both a manual initialisation and tracking to compose a volume, hindering clinical translation. In this paper, we extend a proposed registration approach using Content-Based Image Retrieval (CBIR), removing the requirement for tracking or manual initialisation. Pre-operatively, a set of possible LUS planes is simulated from CT and a descriptor generated for each image. Then, a Bayesian framework is employed to estimate the most likely sequence of CT simulations that matches a series of LUS images. We extend our CBIR formulation to use multiple labelled objects and constrain the registration by separating liver vessels into portal vein and hepatic vein branches. The value of this new labeled approach is demonstrated in retrospective data from 5 patients. Results show that, by including a series of 5 untracked images in time, a single LUS image can be registered with accuracies ranging from 5.7 to 16.4 mm with a success rate of 78%. Initialisation of the LUS to CT registration with the proposed framework could potentially enable the clinical translation of these image fusion techniques.

Are fiducial registration error and target registration error correlated? SciKit-SurgeryFRED for teaching and research

Understanding the relationship between fiducial registration error (FRE) and target registration error (TRE) is important for the correct use of interventional guidance systems. Whilst it is well established that TRE is statistically independent of FRE, system users still struggle against the intuitive assumption that a low FRE indicates a low TRE. We present the SciKit-Surgery Fiducial Registration Educational Demonstrator and describe its use. SciKit-SurgeryFRED was developed to enable remote teaching of key concepts in image registration. SciKit-SurgeryFRED also supports research into user interface design for image registration systems. SciKit-SurgeryFRED can be used to enable remote tutorials covering the statistics relevant to image guided interventions. Students are able to place fiducial markers on pre and intra-operative images and observe the effects of changes in marker geometry, marker count, and fiducial localisation error on TRE and FRE. SciKit-SurgeryFRED also calculates statistical measures for the expected values of TRE and FRE. Because many registrations can be performed quickly the students can then explore potential correlations between the different statistics. SciKit-SurgeryFRED also implements a registration based game, where participants are rewarded for complete treatment of a clinical target, whilst minimising the treatment margin. We used this game to perform a remote study on registration and simulated ablation, measuring how user performance changes depending on what error statistics are made available. The results support the assumption that knowing the exact value of target registration error leads to better treatment. Display of other statistics did not have a significant impact on the treatment performance.

Machine Learning for Localizing Epileptogenic-Zone in the Temporal Lobe: Quantifying the Value of Multimodal Clinical-Semiology and Imaging Concordance

Background: Epilepsy affects 50 million people worldwide and a third are refractory to medication. If a discrete cerebral focus or network can be identified, neurosurgical resection can be curative. Most excisions are in the temporal-lobe, and are more likely to result in seizure-freedom than extra-temporal resections. However, less than half of patients undergoing surgery become entirely seizure-free. Localizing the epileptogenic-zone and individualized outcome predictions are difficult, requiring detailed evaluations at specialist centers. Methods: We used bespoke natural language processing to text-mine 3,800 electronic health records, from 309 epilepsy surgery patients, evaluated over a decade, of whom 126 remained entirely seizure-free. We investigated the diagnostic performances of machine learning models using set-of-semiology (SoS) with and without hippocampal sclerosis (HS) on MRI as features, using STARD criteria. Findings: Support Vector Classifiers (SVC) and Gradient Boosted (GB) decision trees were the best performing algorithms for temporal-lobe epileptogenic zone localization (cross-validated Matthews correlation coefficient (MCC) SVC 0.73 ± 0.25, balanced accuracy 0.81 ± 0.14, AUC 0.95 ± 0.05). Models that only used seizure semiology were not always better than internal benchmarks. The combination of multimodal features, however, enhanced performance metrics including MCC and normalized mutual information (NMI) compared to either alone (p < 0.0001). This combination of semiology and HS on MRI increased both cross-validated MCC and NMI by over 25% (NMI, SVC SoS: 0.35 ± 0.28 vs. SVC SoS+HS: 0.61 ± 0.27). Interpretation: Machine learning models using only the set of seizure semiology (SoS) cannot unequivocally perform better than benchmarks in temporal epileptogenic-zone localization. However, the combination of SoS with an imaging feature (HS) enhance epileptogenic lobe localization. We quantified this added NMI value to be 25% in absolute terms. Despite good performance in localization, no model was able to predict seizure-freedom better than benchmarks. The methods used are widely applicable, and the performance enhancements by combining other clinical, imaging and neurophysiological features could be similarly quantified. Multicenter studies are required to confirm generalizability. Funding: Wellcome/EPSRC Center for Interventional and Surgical Sciences (WEISS) (203145Z/16/Z).

An Enhanced Visualization of DBT Imaging Using Blind Deconvolution and Total Variation Minimization Regularization

Digital Breast Tomosynthesis (DBT) presents out-of-plane artifacts caused by features of high intensity. Given observed data and knowledge about the point spread function (PSF), deconvolution techniques recover data from a blurred version. However, a correct PSF is difficult to achieve and these methods amplify noise. When no information is available about the PSF, blind deconvolution can be used. Additionally, Total Variation (TV) minimization algorithms have achieved great success due to its virtue of preserving edges while reducing image noise. This work presents a novel approach in DBT through the study of out-of-plane artifacts using blind deconvolution and noise regularization based on TV minimization. Gradient information was also included. The methodology was tested using real phantom data and one clinical data set. The results were investigated using conventional 2D slice-by-slice visualization and 3D volume rendering. For the 2D analysis, the artifact spread function (ASF) and Full Width at Half Maximum (FWHMM_ASF) of the ASF were considered. The 3D quantitative analysis was based on the FWHM of disks profiles at 90°, noise and signal to noise ratio (SNR) at 0° and 90°. A marked visual decrease of the artifact with reductions of FWHM_ASF (2D) and FWHM_90° (volume rendering) of 23.8% and 23.6%, respectively, was observed. Although there was an expected increase in noise level, SNR values were preserved after deconvolution. Regardless of the methodology and visualization approach, the objective of reducing the out-of-plane artifact was accomplished. Both for the phantom and clinical case, the artifact reduction in the z was markedly visible.

Impact of total variation minimization in volume rendering visualization of breast tomosynthesis data

BACKGROUND AND OBJECTIVE

Total Variation (TV) minimization algorithms have achieved great attention due to the virtue of decreasing noise while preserving edges. The purpose of this work is to implement and evaluate two TV minimization methods in 3D. Their performance is analyzed through 3D visualization of digital breast tomosynthesis (DBT) data with volume rendering.

METHODS

Both filters were studied with real phantom and one clinical DBT data. One algorithm was applied sequentially to all slices and the other was applied to the entire volume at once. The suitable Lagrange multiplier used in each filter equation was studied to reach the minimum 3D TV and the maximum contrast-to-noise ratio (CNR). Imaging blur was measured at 0° and 90° using two disks with different diameters (0.5 mm and 5.0 mm) and equal thickness. The quality of unfiltered and filtered data was analyzed with volume rendering at 0° and 90°.

RESULTS

For phantom data, with the sequential filter, a decrease of 25% in 3D TV value and an increase of 19% and 30% in CNR at 0° and 90°, respectively, were observed. When the filter is applied directly in 3D, TV value was reduced by 35% and an increase of 36% was achieved both for CNR at 0° and 90°. For the smaller disk, variations of 0% in width at half maximum (FWHM) at 0° and a decrease of about 2.5% for FWHM at 90° were observed for both filters. For the larger disk, there was a 2.5% increase in FWHM at 0° for both filters and a decrease of 6.28% and 1.69% in FWHM at 90° with the sequential filter and the 3D filter, respectively. When applied to clinical data, the performance of each filter was consistent with that obtained with the phantom.

CONCLUSIONS

Data analysis confirmed the relevance of these methods in improving quality of DBT images. Additionally, this type of 3D visualization showed that it may play an important complementary role in DBT imaging. It allows to visualize all DBT data at once and to analyze properly filters applied to all the three dimensions. Concise Abstract Total Variation (TV) minimization algorithms are one compressed sensing technique that has achieved great attention due to the virtue of decrease noise while preserve edges transitions. The purpose of this work is to solve the same TV minimization problem in DBT data, by studying two 3D filters. The obtained results were analyzed at 0° and 90° with a 3D visualization through volume rendering. The filters differ in their application. One considers a slice-by-slice optimization, sequentially traversing all slices of the data. The other considers the intensity values of adjacent slices to make this optimization on each voxel. The performance of each filter was also tested with a clinical case. The results obtained were very encouraging with a significantly increased contrast to noise ratio at 0° and 90° and a small reduction in blur at 90° (slight reduction of the out-of-plane artifact).

Optimization of Breast Tomosynthesis Visualization through 3D Volume Rendering

3D volume rendering may represent a complementary option in the visualization of Digital Breast Tomosynthesis (DBT) examinations by providing an understanding of the underlying data at once. Rendering parameters directly influence the quality of rendered images. The purpose of this work is to study the influence of two of these parameters (voxel dimension in z direction and sampling distance) on DBT rendered data. Both parameters were studied with a real phantom and one clinical DBT data set. The voxel size was changed from 0.085 × 0.085 × 1.0 mm³ to 0.085 × 0.085 × 0.085 mm³ using ten interpolation functions available in the Visualization Toolkit library (VTK) and several sampling distance values were evaluated. The results were investigated at 90º using volume rendering visualization with composite technique. For phantom quantitative analysis, degree of smoothness, contrast-to-noise ratio, and full width at half maximum of a Gaussian curve fitted to the profile of one disk were used. Additionally, the time required for each visualization was also recorded. Hamming interpolation function presented the best compromise in image quality. The sampling distance values that showed a better balance between time and image quality were 0.025 mm and 0.05 mm. With the appropriate rendering parameters, a significant improvement in rendered images was achieved.

SciKit-Surgery: compact libraries for surgical navigation

Purpose

This paper introduces the SciKit-Surgery libraries, designed to enable rapid development of clinical applications for image-guided interventions. SciKit-Surgery implements a family of compact, orthogonal, libraries accompanied by robust testing, documentation, and quality control. SciKit-Surgery libraries can be rapidly assembled into testable clinical applications and subsequently translated to production software without the need for software reimplementation. The aim is to support translation from single surgeon trials to multicentre trials in under 2 years.

Methods

At the time of publication, there were 13 SciKit-Surgery libraries provide functionality for visualisation and augmented reality in surgery, together with hardware interfaces for video, tracking, and ultrasound sources. The libraries are stand-alone, open source, and provide Python interfaces. This design approach enables fast development of robust applications and subsequent translation. The paper compares the libraries with existing platforms and uses two example applications to show how SciKit-Surgery libraries can be used in practice.

Results

Using the number of lines of code and the occurrence of cross-dependencies as proxy measurements of code complexity, two example applications using SciKit-Surgery libraries are analysed. The SciKit-Surgery libraries demonstrate ability to support rapid development of testable clinical applications. By maintaining stricter orthogonality between libraries, the number, and complexity of dependencies can be reduced. The SciKit-Surgery libraries also demonstrate the potential to support wider dissemination of novel research.

Conclusion

The SciKit-Surgery libraries utilise the modularity of the Python language and the standard data types of the NumPy package to provide an easy-to-use, well-tested, and extensible set of tools for the development of applications for image-guided interventions. The example application built on SciKit-Surgery has a simpler dependency structure than the same application built using a monolithic platform, making ongoing clinical translation more feasible.

Learning from irregularly sampled data for endomicroscopy super-resolution: a comparative study of sparse and dense approaches

Purpose

Probe-based confocal laser endomicroscopy (pCLE) enables performing an optical biopsy via a probe. pCLE probes consist of multiple optical fibres arranged in a bundle, which taken together generate signals in an irregularly sampled pattern. Current pCLE reconstruction is based on interpolating irregular signals onto an over-sampled Cartesian grid, using a naive linear interpolation. It was shown that convolutional neural networks (CNNs) could improve pCLE image quality. Yet classical CNNs may be suboptimal in regard to irregular data.

Methods

We compare pCLE reconstruction and super-resolution (SR) methods taking irregularly sampled or reconstructed pCLE images as input. We also propose to embed a Nadaraya–Watson (NW) kernel regression into the CNN framework as a novel trainable CNN layer. We design deep learning architectures allowing for reconstructing high-quality pCLE images directly from the irregularly sampled input data. We created synthetic sparse pCLE images to evaluate our methodology.

Results

The results were validated through an image quality assessment based on a combination of the following metrics: peak signal-to-noise ratio and the structural similarity index. Our analysis indicates that both dense and sparse CNNs outperform the reconstruction method currently used in the clinic.

Conclusion

The main contributions of our study are a comparison of sparse and dense approach in pCLE image reconstruction. We also implement trainable generalised NW kernel regression as a novel sparse approach. We also generated synthetic data for training pCLE SR.

Electronic supplementary material

The online version of this article (10.1007/s11548-020-02170-7) contains supplementary material, which is available to authorized users.

Is there rationale for the cuff pressures prescribed for blood flow restriction exercise? A systematic review

BACKGROUND

Blood flow restriction exercise has increasingly broad applications among healthy and clinical populations. Ensuring the technique is applied in a safe, controlled, and beneficial way for target populations is essential. Individualized cuff pressures are a favored method for achieving this. However, there remains marked inconsistency in how individualized cuff pressures are applied.

OBJECTIVES

To quantify the cuff pressures used in the broader blood flow restriction exercise literature, and determine whether there is clear justification for the choice of pressure prescribed.

METHODS

Studies were included in this review from database searches if they employed an experimental design using original data, involved either acute or chronic exercise using blood flow restriction, and they assessed limb or arterial occlusion pressure to determine an individualized cuff pressure. Methodologies of the studies were evaluated using a bespoke quality assessment tool.

RESULTS

Fifty-one studies met the inclusion criteria. Individualized cuff pressures ranged from 30% to 100% arterial occlusion pressure. Only 7 out of 52 studies attempted to justify the individualized cuff pressure applied during exercise. The mean quality rating for all studies was 11.1 ± 1.2 out of 13.

CONCLUSIONS

The broader blood flow restriction exercise literature uses markedly heterogeneous prescription variables despite using individualized cuff pressures. This is problematic in the absence of any clear justification for the individualized cuff pressures selected. Systematically measuring and reporting all relevant acute responses and training adaptations to the full spectrum of BFR pressures alongside increased clarity around the methodology used during blood flow restriction exercise is paramount.

SnappySonic: An Ultrasound Acquisition Replay Simulator

SnappySonic provides an ultrasound acquisition replay simulator designed for public engagement and training. It provides a simple interface to allow users to experience ultrasound acquisition without the need for specialist hardware or acoustically compatible phantoms. The software is implemented in Python, built on top of a set of open source Python modules targeted at surgical innovation. The library has high potential for reuse, most obviously for those who want to simulate ultrasound acquisition, but it could also be used as a user interface for displaying high dimensional images or video data.

Hemodynamic and perceptual responses to blood flow-restricted exercise among patients undergoing dialysis

End-stage kidney disease is associated with reduced exercise capacity, muscle atrophy, and impaired muscle function. While these may be improved with exercise, single modalities of exercise do not traditionally elicit improvements across all required physiological domains. Blood flow-restricted exercise may improve all of these physiological domains with low intensities traditionally considered insufficient for these adaptions. Investigation of this technique appeals, but is yet to be evaluated, in patients undergoing dialysis. With the use of a progressive crossover design, 10 satellite patients undergoing hemodialysis underwent three exercise conditions over 2 wk: two bouts (10 min) of unrestricted cycling during two consecutive hemodialysis sessions (condition 1), two bouts of cycling with blood flow restriction while off hemodialysis on 2 separate days (condition 2), and two bouts of cycling with blood flow restriction during two hemodialysis sessions (condition 3). Outcomes included hemodynamic responses (heart rate and blood pressure) throughout all sessions, participant-perceived exertion and discomfort on a Borg scale, and evaluation of ultrafiltration rates and dialysis adequacy (Kt/V) obtained post hoc. Hemodynamic responses were consistent regardless of condition. Significant increases in heart rate, systolic blood pressure, and mean arterial blood pressure (P < 0.05) were observed postexercise followed by a reduction in blood pressures during the 60-min recovery (12, 5, and 11 mmHg for systolic, diastolic, and mean arterial pressures, respectively). Blood pressures returned to predialysis ranges following the recovery period. Blood flow restriction did not affect ultrafiltration achieved or Kt/V. Hemodynamic safety and tolerability of blood flow restriction during aerobic exercise on hemodialysis is comparable to standard aerobic exercise.

Spectral Imaging Of Thermal Damage Induced During Microwave Ablation In The Liver

Induction of thermal damage to tissue through delivery of microwave energy is frequently applied in surgery to destroy diseased tissue such as cancer cells. Minimization of unwanted harm to healthy tissue is still achieved subjectively, and the surgeon has few tools at their disposal to monitor the spread of the induced damage. This work describes the use of optical methods to monitor the time course of changes to the tissue during delivery of microwave energy in the porcine liver. Multispectral imaging and diffuse reflectance spectroscopy are used to monitor temporal changes in optical properties in parallel with thermal imaging. The results demonstrate the ability to monitor the spatial extent of thermal damage on a whole organ, including possible secondary effects due to vascular damage. Future applications of this type of imaging may see the multispectral data used as a feedback mechanism to avoid collateral damage to critical healthy structures and to potentially verify sufficient application of energy to the diseased tissue.

VEROnA Protocol: A Pilot, Open-Label, Single-Arm, Phase 0, Window-of-Opportunity Study of Vandetanib-Eluting Radiopaque Embolic Beads (BTG-002814) in Patients With Resectable Liver Malignancies

BACKGROUND

Transarterial chemoembolization (TACE) is the current standard of care for patients with intermediate-stage hepatocellular carcinoma (HCC) and is also a treatment option for patients with liver metastases from colorectal cancer. However, TACE is not a curative treatment, and tumor progression occurs in more than half of the patients treated. Despite advances and technical refinements of TACE, including the introduction of drug-eluting beads-TACE, the clinical efficacy of TACE has not been optimized, and improved arterial therapies are required.

OBJECTIVE

The primary objectives of the VEROnA study are to evaluate the safety and tolerability of vandetanib-eluting radiopaque embolic beads (BTG-002814) in patients with resectable liver malignancies and to determine concentrations of vandetanib and the N-desmethyl metabolite in plasma and resected liver following treatment with BTG-002814.

METHODS

The VEROnA study is a first-in-human, open-label, single-arm, phase 0, window-of-opportunity study of BTG-002814 (containing 100 mg vandetanib) delivered transarterially, 7 to 21 days before surgery in patients with resectable liver malignancies. Eligible patients have a diagnosis of colorectal liver metastases, or HCC (Childs Pugh A), diagnosed histologically or radiologically, and are candidates for liver surgery. All patients are followed up for 28 days following surgery. Secondary objectives of this study are to evaluate the anatomical distribution of BTG-002814 on noncontrast-enhanced imaging, to evaluate histopathological features in the surgical specimen, and to assess changes in blood flow on dynamic contrast-enhanced magnetic resonance imaging following treatment with BTG-002814. Exploratory objectives of this study are to study blood biomarkers with the potential to identify patients likely to respond to treatment and to correlate the distribution of BTG-002814 on imaging with pathology by 3-dimensional modeling.

RESULTS

Enrollment for the study was completed in February 2019. Results of a planned interim analysis were reviewed by a safety committee after the first 3 patients completed follow-up. The recommendation of the committee was to continue the study without any changes to the dose or trial design, as there were no significant unexpected toxicities related to BTG-002814.

CONCLUSIONS

The VEROnA study is studying the feasibility of administering BTG-002814 to optimize the use of this novel technology as liver-directed therapy for patients with primary and secondary liver cancer.

TRIAL REGISTRATION

ClinicalTrial.gov NCT03291379; https://clinicaltrials.gov/ct2/show/NCT03291379.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/13696.

Chronic Blood Flow Restriction Exercise Improves Objective Physical Function: A Systematic Review

Background: Blood flow restriction or KAATSU exercise training is associated with greater muscle mass and strength increases than non-blood flow restriction equivalent exercise. Blood flow restriction exercise has been proposed as a possible alternative to more physically demanding exercise prescriptions (such as high-load/high-intensity resistance training) in a range of clinical and chronic disease populations. While the maintenance of muscle mass and size with reduced musculoskeletal tissue loading appeals in many of these physically impaired populations, there remains a disconnect between some of the desired clinical measures for chronic disease populations and those commonly measured in the literature examining blood flow restriction exercise. While strength does play a vital role in physical function, task-specific objective measures of physical function indicative of activities of daily living are often more clinically relevant and applicable for evaluating the success of medical and surgical interventions or monitoring age- and disease-related physical decline.

Objective: To determine whether exercise interventions utilizing blood flow restriction are able to improve objective measures of physical function indicative of activities of daily living.

Methods: A systematic search of Medline, Embase, CINAHL, SPORTDiscus, and Springer identified 13 randomized control trials utilizing an exercise intervention combined with blood flow restriction, while measuring at least one objective measure of physical function. Participants were ≥18 years of age. Systematic review of the literature and quality assessment of the included studies used the Cochrane Collaboration's tool for assessing risk bias.

Results: Data from 13 studies with a total of 332 participants showed blood flow restriction exercise, regardless of modality, most notably increased performance on the 30 s sit-to-stand and timed up and go tests, and generally improved physical function on other tests including walking tests, variations of sit-to-stand tests, and balance, jumping, and stepping tests.

Conclusions: From the evidence available, blood flow restriction exercise of multiple modalities improved objective measures of physical function indicative of activities of daily living.

Muscular Adaptations to Whole Body Blood Flow Restriction Training and Detraining

Resistance training with blood flow restriction is typically performed during single exercises for the lower- or upper-body, which may not replicate real world programming. The present study examined the change in muscle strength and mass in a young healthy population during an 8-week whole body resistance training program, as well as monitoring these adaptations following a 4-week detraining period. Thirty-nine participants (27 males, 12 females) were allocated into four groups: blood flow restriction training (BFR-T); moderate-heavy load training (HL-T), light-load training (LL-T) or a non-exercise control (CON). Testing measurements were taken at Baseline, during mid-point of training (week 4), end of training (week 8) and following four weeks of detraining (week 12) and included anthropometrics, body composition, muscle thickness (MTH) at seven sites, and maximal dynamic strength (1RM) for six resistance exercises. Whole body resistance training with BFR significantly improved lower- and upper-body strength (overall; 11% increase in total tonnage), however, this was similar to LL-T (12%), but both groups were lower in comparison with HL-T (21%) and all groups greater than CON. Some markers of body composition (e.g., lean mass) and MTH significantly increased over the course of the 8-week training period, but these were similar across all groups. Following detraining, whole body strength remained significantly elevated for both BFR-T (6%) and HL-T (14%), but only the HL-T group remained higher than all other groups. Overall, whole body resistance training with blood flow restriction was shown to be an effective training mode to increase muscular strength and mass. However, traditional moderate-heavy load resistance training resulted in greater adaptations in muscle strength and mass as well as higher levels of strength maintenance following detraining.

Exercise interventions for improving objective physical function in patients with end-stage kidney disease on dialysis: a systematic review and meta-analysis

Patients with end-stage kidney disease on dialysis have increased mortality and reduced physical activity, contributing to impaired physical function. Although exercise programs have demonstrated a positive effect on physiological outcomes such as cardiovascular function and strength, there is a reduced focus on physical function. The aim of this review was to determine whether exercise programs improve objective measures of physical function indicative of activities of daily living for patients with end-stage kidney disease on dialysis. A systematic search of Medline, Embase, the Cochrane Central Register of Controlled Trials, and Cumulative Index to Nursing and Allied Health Literature identified 27 randomized control trials. Only randomized control trials using an exercise intervention or significant muscular activation in the intervention, a usual care, nonexercising control group, and at least one objective measure of physical function were included. Participants were ≥18 yr of age, with end-stage kidney disease, undergoing hemo- or peritoneal dialysis. Systematic review of the literature and quality assessment of the included studies used the Cochrane Collaboration’s tool for assessing risk bias. A meta-analysis was completed for the 6-min walk test. Data from 27 studies with 1,156 participants showed that exercise, regardless of modality, generally increased 6-min walk test distance, sit-to-stand time or repetitions, and grip strength as well as step and stair climb times or repetitions, dynamic mobility, and short physical performance battery scores. From the evidence available, exercise, regardless of modality, improved objective measures of physical function for end-stage kidney disease patients undergoing dialysis. It is acknowledged that further well-designed randomized control trials are required.

Electromagnetic tracking in image-guided laparoscopic surgery: Comparison with optical tracking and feasibility study of a combined laparoscope and laparoscopic ultrasound system

PURPOSE

In image-guided laparoscopy, optical tracking is commonly employed, but electromagnetic (EM) systems have been proposed in the literature. In this paper, we provide a thorough comparison of EM and optical tracking systems for use in image-guided laparoscopic surgery and a feasibility study of a combined, EM-tracked laparoscope and laparoscopic ultrasound (LUS) image guidance system.

METHODS

We first assess the tracking accuracy of a laparoscope with two optical trackers tracking retroreflective markers mounted on the shaft and an EM tracker with the sensor embedded at the proximal end, using a standard evaluation plate. We then use a stylus to test the precision of position measurement and accuracy of distance measurement of the trackers. Finally, we assess the accuracy of an image guidance system comprised of an EM-tracked laparoscope and an EM-tracked LUS probe.

RESULTS

In the experiment using a standard evaluation plate, the two optical trackers show less jitter in position and orientation measurement than the EM tracker. Also, the optical trackers demonstrate better consistency of orientation measurement within the test volume. However, their accuracy of measuring relative positions decreases significantly with longer distances whereas the EM tracker's performance is stable; at 50 mm distance, the RMS errors for the two optical trackers are 0.210 and 0.233 mm, respectively, and it is 0.214 mm for the EM tracker; at 250 mm distance, the RMS errors for the two optical trackers become 1.031 and 1.178 mm, respectively, while it is 0.367 mm for the EM tracker. In the experiment using the stylus, the two optical trackers have RMS errors of 1.278 and 1.555 mm in localizing the stylus tip, and it is 1.117 mm for the EM tracker. Our prototype of a combined, EM-tracked laparoscope and LUS system using representative calibration methods showed a RMS point localization error of 3.0 mm for the laparoscope and 1.3 mm for the LUS probe, the lager error of the former being predominantly due to the triangulation error when using a narrow-baseline stereo laparoscope.

CONCLUSIONS

The errors incurred by optical trackers, due to the lever-arm effect and variation in tracking accuracy in the depth direction, would make EM-tracked solutions preferable if the EM sensor is placed at the proximal end of the laparoscope.

Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks

Automatic segmentation of abdominal anatomy on computed tomography (CT) images can support diagnosis, treatment planning, and treatment delivery workflows. Segmentation methods using statistical models and multi-atlas label fusion (MALF) require inter-subject image registrations, which are challenging for abdominal images, but alternative methods without registration have not yet achieved higher accuracy for most abdominal organs. We present a registration-free deep-learning-based segmentation algorithm for eight organs that are relevant for navigation in endoscopic pancreatic and biliary procedures, including the pancreas, the gastrointestinal tract (esophagus, stomach, and duodenum) and surrounding organs (liver, spleen, left kidney, and gallbladder). We directly compared the segmentation accuracy of the proposed method to the existing deep learning and MALF methods in a cross-validation on a multi-centre data set with 90 subjects. The proposed method yielded significantly higher Dice scores for all organs and lower mean absolute distances for most organs, including Dice scores of 0.78 versus 0.71, 0.74, and 0.74 for the pancreas, 0.90 versus 0.85, 0.87, and 0.83 for the stomach, and 0.76 versus 0.68, 0.69, and 0.66 for the esophagus. We conclude that the deep-learning-based segmentation represents a registration-free method for multi-organ abdominal CT segmentation whose accuracy can surpass current methods, potentially supporting image-guided navigation in gastrointestinal endoscopy procedures.