Robot-assisted ultrasound reconstruction for spine surgery: from bench-top to pre-clinical study

3D ultrasound shape completion and anatomical feature detection for minimally invasive spine surgery

Ultrasound (US) with 3D reconstruction is being explored to offer a radiation-free approach to visualizing anatomical structures. Such a method could be useful for navigating and assisting minimally invasive spine surgery where direct sight on the surgical site is absent. During surgery, the pre-operative CT model and surgical plans are registered to the patient's anatomy by using intra-operative US reconstruction. However, accurate and automatic registration remains challenging. This difficulty arises from an incomplete detection of the bone geometry in US images and the challenges in identifying anatomical landmarks. To address the problem, this work presents a pipeline to automate the workflow by offering an initial CT-to-US registration. This work utilizes PointAttN for 3D shape completion that completes occluded bone structures from partial US reconstruction. This enriched point cloud is then segmented using PointNet++ to identify specific anatomical features. To train the network, synthetic 3D representations of partial views are generated from fifty CT models of the lumbar spine by simulating US physics, effectively mimicking the intraoperative scenario. The proposed work yields a mean registration error of 1.34 mm and 1.63 mm on real US reconstructions of agar phantoms and an ex vivo human spine, respectively. This comprehensive 3D representation enhances anatomical feature interpretation, enabling robust automatic registration. The clinical potential of this framework merits further investigation in pre-clinical trials.

Beyond traditional orthopaedic data analysis: AI, multimodal models and continuous monitoring

Multimodal artificial intelligence (AI) has the potential to revolutionise healthcare by enabling the simultaneous processing and integration of various data types, including medical imaging, electronic health records, genomic information and real-time data. This review explores the current applications and future potential of multimodal AI across healthcare, with a particular focus on orthopaedic surgery. In presurgical planning, multimodal AI has demonstrated significant improvements in diagnostic accuracy and risk prediction, with studies reporting an Area under the receiving operator curve presenting good to excellent performance across various orthopaedic conditions. Intraoperative applications leverage advanced imaging and tracking technologies to enhance surgical precision, while postoperative care has been advanced through continuous patient monitoring and early detection of complications. Despite these advances, significant challenges remain in data integration, standardisation, and privacy protection. Technical solutions such as federated learning (allowing decentralisation of models) and edge computing (allowing data analysis to happen on site or closer to site instead of multipurpose datacenters) are being developed to address these concerns while maintaining compliance with regulatory frameworks. As this field continues to evolve, the integration of multimodal AI promises to advance personalised medicine, improve patient outcomes, and transform healthcare delivery through more comprehensive and nuanced analysis of patient data. Level of Evidence: Level V.

A method for mimicking tumour tissue in brain ex-vivo ultrasound for research application and clinical training

BACKGROUND

Intraoperative ultrasound is becoming a common tool in neurosurgery. However, effective simulation methods are limited. Current, commercial, and homemade phantoms lack replication of anatomical correctness and texture complexity of brain and tumour tissue in ultrasound images.

METHODS

We utilise ex-vivo brain tissue, as opposed to synthetic materials, to achieve realistic echogenic complexity and anatomical correctness. Agar, at 10-20% concentrate, is injected into brain tissue to simulate the tumour mass. A commercially available phantom was purchased for benchmarking.

RESULTS

Qualitative analysis is performed by experienced professionals, measuring the impact of the addition of agar and comparing it to the commercial phantom. Overall, the use of ex vivo tissue was deemed more accurate and representative, compared to the synthetic materials-based phantom, as it provided good visualisation of real brain anatomy and good contrast within tissue. The agar tumour correctly produced a region of higher echogenicity with slight diffusion along the margin and expected interaction with the neighbouring anatomy.

CONCLUSION

The proposed method for creating tumour-mimicking tissue in brain tissue is inexpensive, accurate, and simple. Beneficial for both the trainee clinician and the researcher. A total of 576 annotated images are made publicly available upon request.

A multi-chamber soft robot for transesophageal echocardiography: continuous kinematic matching control of soft medical robots

OBJECTIVES

This research investigates designing a continuum soft robot and proposing a kinematic matching control to enable the robot to perform a specified medical task, which in this paper is the transesophageal echocardiography (TEE).

METHODS

A multi-chamber soft robot was designed and fabricated based on the molding of separate layers. The method of transformation matrices was used to develop the kinematic models, and a control method using Jacobian matrices was proposed to manipulate the robot.

RESULTS

A prototype was made based on a multi-chamber multi-layer design. The system contains three segments that can be actuated independently to mimic the active bending part of the respective probe. Kinematic models were developed. Negative pressure (vacuum) was used as actuation input. An open-loop controller inspired by a redundancy resolution technique was proposed to make the soft robot tip follow the desired path, i.e. the path of the rigid ultrasound probe.

CONCLUSIONS

It is concluded that the soft solution can perform the required task as the reachable points of the TEE tip cover the proposed robot workspace and the proposed control can be used for maneuvering in arbitrary trajectories.

Ultrasound-based 3D bone modelling in computer assisted orthopedic surgery - a review and future challenges

Computer-assisted orthopedic surgery requires precise representations of bone surfaces. To date, computed tomography constitutes the gold standard, but comes with a number of limitations, including costs, radiation and availability. Ultrasound has potential to become an alternative to computed tomography, yet suffers from low image quality and limited field-of-view. These shortcomings may be addressed by a fully automatic segmentation and model-based completion of 3D bone surfaces from ultrasound images. This survey summarizes the state-of-the-art in this field by introducing employed algorithms, and determining challenges and trends. For segmentation, a clear trend toward machine learning-based algorithms can be observed. For 3D bone model completion however, none of the published methods involve machine learning. Furthermore, data sets and metrics are identified as weak spots in current research, preventing development and evaluation of models that generalize well.

Development of Automatic Assessment Framework for Spine Deformity Using Freehand 3-D Ultrasound Imaging System

A 3-D ultrasound (US) imaging technique has been studied to facilitate the diagnosis of spinal deformity without radiation. The objective of this article is to propose an assessment framework to automatically estimate spinal deformity in US spine images. The proposed framework comprises four major components, a US spine image generator, a novel transformer-based lightweight spine detector network, an angle evaluator, and a 3-D modeler. The principal component analysis (PCA) and discriminative scale space tracking (DSST) method are first adopted to generate the US spine images. The proposed detector is equipped with a redundancy queries removal (RQR) module and a regularization item to realize accurate and unique detection of spine images. Two clinical datasets, a total of 273 images from adolescents with idiopathic scoliosis, are used for the investigation of the proposed framework. The curvature is estimated by the angle evaluator, and the 3-D mesh model is established by the parametric modeling technique. The accuracy rate (AR) of the proposed detector can be achieved at 99.5%, with a minimal redundancy rate (RR) of 1.5%. The correlations between automatic curve measurements on US spine images from two datasets and manual measurements on radiographs are 0.91 and 0.88, respectively. The mean absolute difference (MAD) and standard deviation (SD) are 2.72° ± 2.14° and 2.91° ± 2.36° , respectively. The results demonstrate the effectiveness of the proposed framework to advance the application of the 3-D US imaging technique in clinical practice for scoliosis mass screening and monitoring.