Safe contact-based robot active search using Bayesian optimization and control barrier functions

In robotics, active exploration and learning in uncertain environments must take into account safety, as the robot may otherwise damage itself or its surroundings. This paper presents a method for safe active search using Bayesian optimization and control barrier functions. As robot paths undertaken during sampling are continuous, we consider an informative continuous expected improvement acquisition function. To safely bound the contact forces between the robot and its surroundings, we leverage exponential control barrier functions, utilizing the derivative of the force in the contact model to increase robustness to uncertainty in the contact boundary. Our approach is demonstrated on a fully autonomous robot for ultrasound scanning of rheumatoid arthritis (RA). Here, active search is a critical component of ensuring high image quality. Furthermore, bounded contact forces between the ultrasound probe and the patient ensure patient safety and better scan quality. To the best of our knowledge, our results are both the first demonstration of safe active search on a fully autonomous robot for ultrasound scanning of rheumatoid arthritis and the first experimental evaluation of bounding contact forces in the context of medical robotics using control barrier functions. The results show that when search time is limited to less than 60 s, informative continuous expected improvement leads to a 92% success, a 13% improvement compared to expected improvement. Meanwhile, exponential control barrier functions can limit the force applied by the robot to under 5 N, even in cases where the contact boundary is specified incorrectly by -1 or +4 mm.

Artificial intelligence model for segmentation and severity scoring of osteophytes in hand osteoarthritis on ultrasound images

Objective

To develop an artificial intelligence (AI) model able to perform both segmentation of hand joint ultrasound images for osteophytes, bone, and synovium and perform osteophyte severity scoring following the EULAR-OMERACT grading system (EOGS) for hand osteoarthritis (OA).

Methods

One hundred sixty patients with pain or reduced function of the hands were included. Ultrasound images of the metacarpophalangeal (MCP), proximal interphalangeal (PIP), distal interphalangeal (DIP), and first carpometacarpal (CMC1) joints were then manually segmented for bone, synovium and osteophytes and scored from 0 to 3 according to the EOGS for OA. Data was divided into a training, validation, and test set. The AI model was trained on the training data to perform bone, synovium, and osteophyte identification on the images. Based on the manually performed image segmentation, an AI was trained to classify the severity of osteophytes according to EOGS from 0 to 3. Percent Exact Agreement (PEA) and Percent Close Agreement (PCA) were assessed on individual joints and overall. PCA allows a difference of one EOGS grade between doctor assessment and AI.

Results

A total of 4615 ultrasound images were used for AI development and testing. The developed AI model scored on the test set for the MCP joints a PEA of 76% and PCA of 97%; for PIP, a PEA of 70% and PCA of 97%; for DIP, a PEA of 59% and PCA of 94%, and CMC a PEA of 50% and PCA of 82%. Combining all joints, we found a PEA between AI and doctor assessments of 68% and a PCA of 95%.

Conclusion

The developed AI model can perform joint ultrasound image segmentation and severity scoring of osteophytes, according to the EOGS. As proof of concept, this first version of the AI model is successful, as the agreement performance is slightly higher than previously found agreements between experts when assessing osteophytes on hand OA ultrasound images. The segmentation of the image makes the AI explainable to the doctor, who can immediately see why the AI applies a given score. Future validation in hand OA cohorts is necessary though.

E-learning and practical performance in musculoskeletal ultrasound: a multicentre randomized study

OBJECTIVES

To examine the effect of pre-course e-learning on residents' practical performance in musculoskeletal ultrasound (MSUS).

METHODS

This was a multicentre, randomized controlled study following the Consolidated Standards of Reporting Trials (CONSORT) statement. Residents with no or little MSUS experience were randomized to either an e-learning group or a traditional group. One week before a 2-day face-to-face MSUS course, the e-learning group received access to an interactive platform consisting of online lectures, assignments, and practical instruction videos aligned with the content of the course. The traditional group only received standard pre-course information (program, venue, and time). All participants performed a pre- and post-course practical MSUS examination and were assessed by two individual raters, blinded to the group allocation, using the validated Objective Structured Assessment of Ultrasound Skills (OSAUS) tool.

RESULTS

Twenty-eight participants completed the study. There were no statistically significant differences in the pre- or post-course practical MSUS performance between the e-learning group and the traditional group; the mean pre-course OSAUS score (s.d.) in the -learning group was 5.4 (3.7) compared with 5.2 (2.4) in the traditional group (P = 0.8), whereas the post-course OSAUS score in the e-learning group was 11.1 (2.8) compared with 10.9 (2.4) in the traditional group (P = 0.8). There was a significant difference between the mean pre- and post-course scores (5.74 points, P < 0.001). The OSAUS assessment tool demonstrated good inter-rater reliability (intra-class correlation = 0.84).

CONCLUSION

We found no significant impact of pre-course e-learning on novices' acquisition of practical MSUS skills. Hands-on training is of the utmost importance and improves MSUS performance significantly. The OSAUS assessment tool is an applicable tool with high interrater reliability.

TRIAL REGISTRATION

https://clinicaltrials.gov/ NCT04959162.

Retinal main vessel calibers and systemic markers for long-term development of proliferative diabetic retinopathy

PURPOSE

To evaluate if retinal vascular calibers and systemic risk factors in patients with no or minimal diabetic retinopathy (DR) can predict risk of long-term progression to proliferative diabetic retinopathy (PDR).

METHODS

This was a matched case-control study of patients with diabetes having no or minimal DR at baseline with (cases) or without (controls) subsequent development of PDR. We collected six-field, 45-degree retinal images, demographic and clinical data from the Funen Diabetes Database.

RESULTS

We included 52 eyes from 39 cases and 107 eyes from 89 controls matched on sex, age, type of diabetes, time from first to last screening episode and baseline DR level. Cases had higher HbA1c (73 vs. 55 mmoL/moL; p < 0.001), triglycerides (1.32 vs. 1.16 mmoL/L; p = 0.02) and longer duration of diabetes (19 vs. 14 years; p = 0.01), but the groups did not differ in calibers of retinal arterioles (229 vs. 227 μm; p = 0.49), venules (289 vs. 290 μm; p = 0.83) or the arterio-to-venule ratio (0.78 vs. 0.77; p = 0.86).In a multivariable logistic regression model with cluster robust standard error, HbA1c (OR 1.54 per 10 mmoL/moL; 95%-CI: 1.15-2.07; p = 0.004), triglyceride (OR 1.39 per 1 mmoL/L; 95%-CI: 1.03-1.86; p = 0.03) and duration of diabetes (OR 1.09 per year; 95%-CI: 1.03-1.16; p = 0.003) were independent risk factors for PDR.

CONCLUSION

Retinal vascular calibers did not predict long-term development of PDR in contrast to well-established risk factors like HbA1c, triglyceride and duration of diabetes.

Five-Year Incidence of Proliferative Diabetic Retinopathy and Associated Risk Factors in a Nationwide Cohort of 201 945 Danish Patients with Diabetes

Purpose

To evaluate the proliferative diabetic retinopathy (PDR) progression rates and identify the demographic and clinical characteristics of patients who later developed PDR compared with patients who did not progress to that state.

Design

A national 5-year register-based cohort study including 201 945 patients with diabetes.

Subjects

Patients with diabetes who had attended the Danish national screening program (2013-2018) for diabetic retinopathy (DR).

Methods

We used the first screening episode as the index date and included both eyes of patients with and without subsequent progression of PDR. Data were linked with various national health registries to investigate relevant clinical and demographic parameters. The International Clinical Retinopathy Disease Scale was used to classify DR, with no DR as level 0, mild DR as level 1, moderate DR as level 2, severe DR as level 3, and PDR as level 4.

Main Outcome Measures

Hazard ratios (HRs) for incident PDR for all relevant demographic and clinical parameters and 1-, 3-, and 5-year incidence rates of PDR according to baseline DR level.

Results

Progression to PDR within 5 years was identified in 2384 eyes of 1780 patients. Proliferative diabetic retinopathy progression rates from baseline DR level 3 at 1, 3 and 5 years were 3.6%, 10.9%, and 14.7%, respectively. The median number of visits was 3 (interquartile range, 1-4). Progression to PDR was predicted in a multivariable model by duration of diabetes (HR, 4.66 per 10 years; 95% confidence interval [CI], 4.05-5.37), type 1 diabetes (HR, 9.61; 95% CI, 8.01-11.53), a Charlson Comorbidity Index score of > 0 (score 1: HR, 4.62; 95% CI, 4.14-5.15; score 2: HR, 2.28; 95% CI, 1.90-2.74; score ≥ 3: HR, 4.28; 95% CI, 3.54-5.17), use of insulin (HR, 5.33; 95% CI, 4.49-6.33), and use of antihypertensive medications (HR, 2.23; 95% CI, 1.90-2.61).

Conclusions

In a 5-year longitudinal study of an entire screening nation, we found increased risk of PDR with increasing baseline DR levels, longer duration of diabetes, type 1 diabetes, systemic comorbidity, use of insulin, and blood pressure-lowering medications. Most interestingly, we found lower risk of progression from DR level 3 to PDR compared with that in previous studies.

Financial Disclosures

Proprietary or commercial disclosure may be found after the references.

Ultrasound joint examination by an automated system versus by a rheumatologist: from a patient perspective

Background

The Arthritis Ultrasound Robot (ARTHUR) is an automated system for ultrasound scanning of the joints of both hands and wrists, with subsequent disease activity scoring using artificial intelligence. The objective was to describe the patient’s perspective of being examined by ARTHUR, compared to an ultrasound examination by a rheumatologist. Further, to register any safety issues with the use of ARTHUR.

Methods

Twenty-five patients with rheumatoid arthritis (RA) had both hands and wrists examined by ultrasound, first by a rheumatologist and subsequently by ARTHUR. Patient-reported outcomes (PROs) were obtained after the examination by the rheumatologist and by ARTHUR. PROs regarding pain, discomfort and overall experience were collected, including willingness to be examined again by ARTHUR as part of future clinical follow-up. All ARTHUR examinations were observed for safety issues.

Results

There was no difference in pain or discomfort between the examination by a rheumatologist and by ARTHUR (p = 0.29 and p = 0.20, respectively). The overall experience of ARTHUR was described as very good or good by 92% (n = 23), with no difference compared to the examination by the rheumatologist (p = 0.50). All (n = 25) patients were willing to be examined by ARTHUR again, and 92% (n = 23) would accept ARTHUR as a regular part of their RA clinical follow up. No safety issues were registered.

Conclusions

Joint ultrasound examination by ARTHUR was safe and well-received, with no difference in PRO components compared to ultrasound examination by a rheumatologist. Fully automated systems for RA disease activity assessment could be important in future strategies for managing RA patients.

Trial registration: The study was evaluated by the regional ethics committee (ID: S-20200145), which ruled it was not a clinical trial necessary for their approval. It was a quality assessment project, as there was no intervention to the patient. The study was hereafter submitted and registered to Odense University Hospital, Region of Southern Denmark as a quality assessment project and approved (ID: 20/55294).

Monopolar, bipolar, tripolar, and tetrapolar configurations in robot assisted electrical impedance scanning

OBJECTIVE

Tissue recognition is a critical process during a Robot-assisted minimally invasive surgery (RMIS) and it relies on the involvement of advanced sensing technology.

APPROACH

In this paper, the concept of Robot Assisted Electrical Impedance Sensing (RAEIS) is utilized and further developed aiming to sense the electrical bioimpedance of target tissue directly based on the existing robotic instruments and control strategy. Specifically, we present a new sensing configuration called pseudo-tetrapolar method. With the help of robotic control, we can achieve a similar configuration as traditional tetrapolar, and with better accuracy.

MAIN RESULTS

Five configurations including monopolar, bipolar, tripolar, tetrapolar and pseudo-tetrapolar are analyzed and compared through simulation experiments. Advantages and disadvantages of each configuration are thus discussed.

SIGNIFICANCE

This study investigates the measurement of tissue electrical property directly based on the existing robotic surgical instruments. Specifically, different sensing configurations can be realized through different connection and control strategies, making them suitable for different application scenarios.

A digital online platform for education and certification of diabetic retinopathy health care professionals in the Region of Southern Denmark

PURPOSE

The incidence of diabetes continues to increase across the world. As the number of patients rises, so does the need for educated health care professionals. Diabetic retinopathy (DR) remains one of the primary complications in diabetes, and screening has proved to be a cost-effective measure to avoid DR-related blindness. Denmark has an established screening programme, but no formal training of the people responsible for analysing retinal images.

METHODS

We here present an online learning platform that offers a diabetic eye screening course for health care professionals undertaking screening responsibility in the Region of Southern Denmark. The course is divided into lectures, each focussed on identifying different levels of DR or detecting related lesions. The course is free to use on-demand, contains instructional videos, interactive tests and exercises, and it is concluded with a certification test. The tools on the platform can in addition be used to generate data for research purposes, such as comparing users or experts in detection of lesions or annotating data for the development of machine learning models.

RESULTS

More than 150 participants have so far completed the course, and the platform is being adopted for education in other regions of Denmark.

A disposable force regulation mechanism for throat swab robot

Robots can protect healthcare workers from being infected by the COVID-19 and play a role in throat swab sampling operation. A critical requirement in this process is to maintain a constant force on the tissue for ensuring a safe and good sampling. In this study, we present the design of a disposable mechanism with two non-linear springs to achieve a 0.6 N constant force within a 20 mm displacement. The nonlinear spring is designed through optimization based on Finite Element Simulation and Genetic Algorithm. Prototype of the mechanism is made and tested. The experimental results show that the mechanism can provide 0.67±0.04 N and 0.57±0.02 N during its compression and return process. The proposed design can be extended to different scales and used in a variety of scenario where safe interacting with human is required.

Robot-Assisted Electrical Impedance Scanning system for 2D Electrical Impedance Tomography tissue inspection

The electrical impedance tomography (EIT) technology is an important medical imaging approach to show the electrical characteristics and the homogeneity of a tissue region noninvasively. Recently, this technology has been introduced to the Robot Assisted Minimally Invasive Surgery (RAMIS) for assisting the detection of surgical margin with relevant clinical benefits. Nevertheless, most EIT technologies are based on a fixed multiple-electrodes probe which limits the sensing flexibility and capability significantly. In this study, we present a method for acquiring the EIT measurements during a RAMIS procedure using two already existing robotic forceps as electrodes. The robot controls the forceps tips to a series of predefined positions for injecting excitation current and measuring electric potentials. Given the relative positions of electrodes and the measured electric potentials, the spatial distribution of electrical conductivity in a section view can be reconstructed. Realistic experiments are designed and conducted to simulate two tasks: subsurface abnormal tissue detection and surgical margin localization. According to the reconstructed images, the system is demonstrated to display the location of the abnormal tissue and the contrast of the tissues' conductivity with an accuracy suitable for clinical applications.

An electrical bioimpedance scanning system for subsurface tissue detection in Robot Assisted Minimally Invasive Surgery

In Robot Assisted Minimally Invasive Surgery, discriminating critical subsurface structures is essential to make the surgical procedure safer and more efficient. In this paper, a novel robot assisted electrical bio-impedance scanning (RAEIS) system is developed and validated using a series of experiments. The proposed system constructs a tri-polar sensing configuration for tissue homogeneity inspection. Specifically, two robotic forceps are used as electrodes for applying electric current and measuring reciprocal voltages relative to a ground electrode which is placed distal from the measuring site. Compared to the other existing electrical bioimpedance sensing technology, the proposed system is able to use miniaturized electrodes to measure a site flexibly with enhanced subsurfacial detection capability. In this paper, we present the concept, the modeling of the sensing method, the hardware design, and the system calibration. Subsequently, a series of experiments are conducted for system evaluation including finite element simulation, saline solution bath experiments and experiments based on ex vivo animal tissues. The experimental results demonstrate that the proposed system can measure the resistivity of the material with high accuracy, and detect a subsurface non-homogeneous object with 100% success rate. The proposed parameters estimation algorithm is able to approximate the resistivity and the depth of the subsurface object effectively with one fast scanning.

Deep learning detects and visualizes bleeding events in electronic health records

BACKGROUND

Bleeding is associated with a significantly increased morbidity and mortality. Bleeding events are often described in the unstructured text of electronic health records, which makes them difficult to identify by manual inspection.

OBJECTIVES

To develop a deep learning model that detects and visualizes bleeding events in electronic health records.

PATIENTS/METHODS

Three hundred electronic health records with International Classification of Diseases, Tenth Revision diagnosis codes for bleeding or leukemia were extracted. Each sentence in the electronic health record was annotated as positive or negative for bleeding. The annotated sentences were used to develop a deep learning model that detects bleeding at sentence and note level.

RESULTS

On a balanced test set of 1178   sentences, the best-performing deep learning model achieved a sensitivity of 0.90, specificity of 0.90, and negative predictive value of 0.90. On a test set consisting of 700 notes, of which 49 were positive for bleeding, the model achieved a note-level sensitivity of 1.00, specificity of 0.52, and negative predictive value of 1.00. By using a sentence-level model on a note level, the model can explain its predictions by visualizing the exact sentence in a note that contains information regarding bleeding. Moreover, we found that the model performed consistently well across different types of bleedings.

CONCLUSIONS

A deep learning model can be used to detect and visualize bleeding events in the free text of electronic health records. The deep learning model can thus facilitate systematic assessment of bleeding risk, and thereby optimize patient care and safety.

Applying cascaded convolutional neural network design further enhances automatic scoring of arthritis disease activity on ultrasound images from rheumatoid arthritis patients

OBJECTIVES

We have previously shown that neural network technology can be used for scoring arthritis disease activity in ultrasound images from rheumatoid arthritis (RA) patients, giving scores according to the EULAR-OMERACT grading system. We have now further developed the architecture of this neural network and can here present a new idea applying cascaded convolutional neural network (CNN) design with even better results. We evaluate the generalisability of this method on unseen data, comparing the CNN with an expert rheumatologist.

METHODS

The images were graded by an expert rheumatologist according to the EULAR-OMERACT synovitis scoring system. CNNs were systematically trained to find the best configuration. The algorithms were evaluated on a separate test data set and compared with the gradings of an expert rheumatologist on a per-joint basis using a Kappa statistic, and on a per-patient basis using a Wilcoxon signed-rank test.

RESULTS

With 1678 images available for training and 322 images for testing the model, it achieved an overall four-class accuracy of 83.9%. On a per-patient level, there was no significant difference between the classifications of the model and of a human expert (p=0.85). Our original CNN had a four-class accuracy of 75.0%.

CONCLUSIONS

Using a new network architecture we have further enhanced the algorithm and have shown strong agreement with an expert rheumatologist on a per-joint basis and on a per-patient basis. This emphasises the potential of using CNNs with this architecture as a strong assistive tool for the objective assessment of disease activity of RA patients.

Surgeons' muscle load during robotic-assisted laparoscopy performed with a regular office chair and the preferred of two ergonomic chairs: A pilot study

Surgeons work in awkward work postures and have high precision demands - well-known risk factors for musculoskeletal pain. Robotic-assisted laparoscopy is expected to be less demanding compared to conventional laparoscopy; however, studies indicate that robotic-assisted laparoscopy is also associated with poor ergonomics and musculoskeletal pain. The ergonomic condition in the robotic console is partially dependent upon the chair provided, which often is a regular office chair. Our study quantified and compared the muscular load during robotic-assisted laparoscopy using one of two custom built ergonomic chairs and a regular office chair. The results demonstrated no differences that could be considered clinically relevant. Overall, the study showed high levels of static and mean muscular activity, increased perceived physical exertion from pre-to-post surgery, and moderate to high risk for musculoskeletal injuries measured by the Rapid Upper Limb Assessment worksheet. Authors advocate for further investigation in surgeons' ergonomics and physical work demands in robotic surgery.

Neural networks for automatic scoring of arthritis disease activity on ultrasound images

Background

The development of standardised methods for ultrasound (US) scanning and evaluation of synovitis activity by the OMERACT-EULAR Synovitis Scoring (OESS) system is a major step forward in the use of US in the diagnosis and monitoring of patients with inflammatory arthritis. The variation in interpretation of disease activity on US images can affect diagnosis, treatment and outcomes in clinical trials. We, therefore, set out to investigate if we could utilise neural network architecture for the interpretation of disease activity on Doppler US images, using the OESS scoring system.

Methods

Two state-of-the-art neural networks were used to extract information from 1342 Doppler US images from patients with rheumatoid arthritis (RA). One neural network divided images as either healthy (Doppler OESS score 0 or 1) or diseased (Doppler OESS score 2 or 3). The other to score images across all four of the OESS systems Doppler US scores (0–3). The neural networks were hereafter tested on a new set of RA Doppler US images (n=176). Agreement between rheumatologist’s scores and network scores was measured with the kappa statistic.

Results

For the neural network assessing healthy/diseased score, the highest accuracies compared with an expert rheumatologist were 86.4% and 86.9% with a sensitivity of 0.864 and 0.875 and specificity of 0.864 and 0.864, respectively. The other neural network developed to four class Doppler OESS scoring achieved an average per class accuracy of 75.0% and a quadratically weighted kappa score of 0.84.

Conclusion

This study is the first to show that neural network technology can be used in the scoring of disease activity on Doppler US images according to the OESS system.