Surrogate optimisation strategies for intraocular lens formula constant optimisation

PURPOSE

To investigate surrogate optimisation (SO) as a modern, purely data-driven, nonlinear adaptive iterative strategy for lens formula constant optimisation in intraocular lens power calculation.

METHODS

A SO algorithm was implemented for optimising the root mean squared formula prediction error (rmsPE, defined as predicted refraction minus achieved refraction) for the SRKT, Hoffer Q, Holladay, Haigis and Castrop formulae in a dataset of N = 888 cataractous eyes with implantation of the Hoya Vivinex hydrophobic acrylic aspheric lens. A Gaussian Process estimator was used as the model, and the SO was initialised with equidistant datapoints within box constraints, and the number of iterations restricted to either 200 (SRKT, Hoffer Q, Holladay) or 700 (Haigis, Castrop). The performance of the algorithm was compared to the classical gradient-based Levenberg-Marquardt algorithm.

RESULTS

The SO algorithm showed stable convergence after fewer than 50/150 iterations (SRKT, HofferQ, Holladay, Haigis, Castrop). The rmsPE was reduced systematically to 0.4407/0.4288/0.4265/0.3711/0.3449 dioptres. The final constants were A = 119.2709, pACD = 5.7359, SF = 1.9688, -a0 = 0.5914/a1 = 0.3570/a2 = 0.1970, C = 0.3171/H = 0.2053/R = 0.0947 for the SRKT, Hoffer Q, Holladay, Haigis and Castrop formula and matched the respective constants optimised in previous studies.

CONCLUSION

The SO proves to be a powerful adaptive nonlinear iteration algorithm for formula constant optimisation, even in formulae with one or more constants. It acts independently of a gradient and is in general able to search within a (box) constrained parameter space for the best solution, even where there are multiple local minima of the target function.

Analysis of microvascular pattern in diabetes mellitus condition using the nailfold capillaroscopy images

Diabetes is often considered a vascular disease due to its impact on blood vessels, it is a complex condition with various metabolic and autoimmune factors involved. One of the long term comorbidities of diabetes includes microvascular complications. The microvascular complications can be analyzed using the Nailfold capillaroscopy, a non-invasive technique that allows for the visualization and analysis of capillaries in the proximal nailfold area. Using advanced video capillaroscopy with high magnification, capillary images can be captured from and processed to analyze their morphology. The capillary images of normal group and diabetic group are acquired from 118 participants using nailfold capillaroscopy and the obtained images are preprocessed using image processing filters. The identification and segmentation of the capillaries are the challenges to be addressed in the processing of the images. Hence segmentation of capillaries is done using morphological operations, thresholding and convolutional neural networks. The performance of the filters and segmentation methods are evaluated using Mean Square Error (MSE), Peak signal to Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), Jaccard Index and Sorensen coefficient. By analyzing the morphological features namely the capillary diameter, density, distribution, presence of hemorrhage and the shape of the capillaries from both the groups, the capillary changes associated with diabetic condition were studied. It was found that the non diabetic participants considered in this study has capillary diameter in the range of 8-14 µm and the capillary density in the range of 10-30 capillaries per mm2 whereas the diabetic participants has capillary diameter greater than 30 µm and the capillary density is less than 10 capillaries per mm2. In addition to capillary density and diameter, the presence of hemorrhage, the orientation and distribution of the capillaries are also considered to differentiate the diabetic group from the non diabetic group. The classification of the participants are validated with the clinical history of the participants.

Human movement modifications induced by different levels of transparency of an active upper limb exoskeleton

Active upper limb exoskeletons are a potentially powerful tool for neuromotor rehabilitation. This potential depends on several basic control modes, one of them being transparency. In this control mode, the exoskeleton must follow the human movement without altering it, which theoretically implies null interaction efforts. Reaching high, albeit imperfect, levels of transparency requires both an adequate control method and an in-depth evaluation of the impacts of the exoskeleton on human movement. The present paper introduces such an evaluation for three different "transparent" controllers either based on an identification of the dynamics of the exoskeleton, or on force feedback control or on their combination. Therefore, these controllers are likely to induce clearly different levels of transparency by design. The conducted investigations could allow to better understand how humans adapt to transparent controllers, which are necessarily imperfect. A group of fourteen participants were subjected to these three controllers while performing reaching movements in a parasagittal plane. The subsequent analyses were conducted in terms of interaction efforts, kinematics, electromyographic signals and ergonomic feedback questionnaires. Results showed that, when subjected to less performing transparent controllers, participants strategies tended to induce relatively high interaction efforts, with higher muscle activity, which resulted in a small sensitivity of kinematic metrics. In other words, very different residual interaction efforts do not necessarily induce very different movement kinematics. Such a behavior could be explained by a natural human tendency to expend effort to preserve their preferred kinematics, which should be taken into account in future transparent controllers evaluation.

Particle swarm optimisation strategies for IOL formula constant optimisation

PURPOSE

To investigate particle swarm optimisation (PSO) as a modern purely data driven non-linear iterative strategy for lens formula constant optimisation in intraocular lens power calculation.

METHODS

A PSO algorithm was implemented for optimising the root mean squared formula prediction error (rmsPE, defined as achieved refraction minus predicted refraction) for the Castrop formula in a dataset of N = 888 cataractous eyes with implantation of the Hoya Vivinex hydrophobic acrylic aspheric lens. The hyperparameters were set to inertia: 0.8, accelerations c1 = c2 = 0.1. The algorithm was initialised with NP  = 100 particles having random positions and velocities within the box constraints of the constant triplet parameter space C = 0.25 to 0.45, H = -0.25 to 0.25 and R = -0.25 to 0.25. The performance of the algorithm was compared to classical gradient-based Trust-Region-Reflective and Interior-Point algorithms.

RESULTS

The PSO algorithm showed fast and stable convergence after 37 iterations. The rmsPE reduced systematically to 0.3440 diopters (D). With further iterations the scatter of the particle positions in the swarm decreased but without further reduction of rmsPE. The final constant triplet was C/H/R = 0.2982/0.2497/0.1435. The Trust-Region-Reflective/Interior-Point algorithms showed convergence after 27/17 iterations, respectively, resulting in formula constant triplets C/H/R = 0.2982/0.2496/0.1436 and 0.2982/0.2495/0.1436, both with the same rmsPE as the PSO algorithm (rmsPE = 0.3440 D).

CONCLUSION

The PSO appears to be a powerful adaptive nonlinear iteration algorithm for formula constant optimisation even in formulae with more than 1 constant. It acts independently of an analytical or numerical gradient and is in general able to search for the best solution even with multiple local minima of the target function.

Digitization of Pathology Labs: A Review of Lessons Learned

Pathology laboratories are increasingly using digital workflows. This has the potential of increasing laboratory efficiency, but the digitization process also involves major challenges. Several reports have been published describing the individual experiences of specific laboratories with the digitization process. However, a comprehensive overview of the lessons learned is still lacking. We provide an overview of the lessons learned for different aspects of the digitization process, including digital case management, digital slide reading, and computer-aided slide reading. We also cover metrics used for monitoring performance and pitfalls and corresponding values observed in practice. The overview is intended to help pathologists, information technology decision makers, and administrators to benefit from the experiences of others and to implement the digitization process in an optimal way to make their own laboratory future-proof.

The Performance of Flash Replenishment Contrast-Enhanced Ultrasound for the Qualitative Assessment of Kidney Lesions in Patients with Chronic Kidney Disease

We investigated the accuracy of CEUS for characterizing cystic and solid kidney lesions in patients with chronic kidney disease (CKD). Cystic lesions are assessed using Bosniak criteria for computed tomography (CT) and magnetic resonance imaging (MRI); however, in patients with moderate to severe kidney disease, CT and MRI contrast agents may be contraindicated. Contrast-enhanced ultrasound (CEUS) is a safe alternative for characterizing these lesions, but data on its performance among CKD patients are limited. We performed flash replenishment CEUS in 60 CKD patients (73 lesions). Final analysis included 53 patients (63 lesions). Four readers, blinded to true diagnosis, interpreted each lesion. Reader evaluations were compared to true lesion classifications. Performance metrics were calculated to assess malignant and benign diagnoses. Reader agreement was evaluated using Bowker's symmetry test. Combined reader sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for diagnosing malignant lesions were 71%, 75%, 45%, and 90%, respectively. Sensitivity (81%) and specificity (83%) were highest in CKD IV/V patients when grouped by CKD stage. Combined reader sensitivity, specificity, PPV, and NPV for diagnosing benign lesions were 70%, 86%, 91%, and 61%, respectively. Again, in CKD IV/V patients, sensitivity (81%), specificity (95%), and PPV (98%) were highest. Inter-reader diagnostic agreement varied from 72% to 90%. In CKD patients, CEUS is a potential low-risk option for screening kidney lesions. CEUS may be particularly beneficial for CKD IV/V patients, where kidney preservation techniques are highly relevant.

The Quantified Surgeon: A Glimpse Into the Future of Surgical Metrics and Outcomes

This paper summarizes key points of the 2023 Southeastern Surgical Congress Laws Lecture. The focus of the presentation was on the use of advanced engineering technology to quantify surgical mastery. New concepts relating to the visual-haptic loop, mastery and perception, and mastery and technical decisions were introduced and shown in an empirical fashion to have relevance in procedural outcomes in a simulated setting. The major takeaway point is that surgical mastery can be quantified using advanced engineering technology, and this process will help to shorten the learning curve.

Exploring the impact of design criteria for reference sets on performance evaluation of signal detection algorithms: The case of drug-drug interactions

PURPOSE

To evaluate the impact of multiple design criteria for reference sets that are used to quantitatively assess the performance of pharmacovigilance signal detection algorithms (SDAs) for drug-drug interactions (DDIs).

METHODS

Starting from a large and diversified reference set for two-way DDIs, we generated custom-made reference sets of various sizes considering multiple design criteria (e.g., adverse event background prevalence). We assessed differences observed in the performance metrics of three SDAs when applied to FDA Adverse Event Reporting System (FAERS) data.

RESULTS

For some design criteria, the impact on the performance metrics was neglectable for the different SDAs (e.g., theoretical evidence associated with positive controls), while others (e.g., restriction to designated medical events, event background prevalence) seemed to have opposing and effects of different sizes on the Area Under the Curve (AUC) and positive predictive value (PPV) estimates.

CONCLUSIONS

The relative composition of reference sets can significantly impact the evaluation metrics, potentially altering the conclusions regarding which methodologies are perceived to perform best. We therefore need to carefully consider the selection of controls to avoid misinterpretation of signals triggered by confounding factors rather than true associations as well as adding biases to our evaluation by "favoring" some algorithms while penalizing others.

Evaluation Methods and Measurement Challenges for Industrial Exoskeletons

In recent years, exoskeleton test methods for industrial exoskeletons have evolved to include simulated laboratory and field environments. Physiological, kinematic, and kinetic metrics, as well as subjective surveys, are used to evaluate exoskeleton usability. In particular, exoskeleton fit and usability can also impact the safety of exoskeletons and their effectiveness at reducing musculoskeletal injuries. This paper surveys the state of the art in measurement methods applied to exoskeleton evaluation. A notional classification of the metrics based on exoskeleton fit, task efficiency, comfort, mobility, and balance is proposed. In addition, the paper describes the test and measurement methods used in supporting the development of exoskeleton and exosuit evaluation methods to assess their fit, usability, and effectiveness in industrial tasks such as peg in hole, load align, and applied force. Finally, the paper includes a discussion of how the metrics can be applied towards a systematic evaluation of industrial exoskeletons, current measurement challenges, and future research directions.

Augmented-reality swim goggles accurately and reliably measure swim performance metrics in recreational swimmers

Background

Swimmers commonly access performance metrics such as lap splits, distance, and pacing information between work bouts while they rest. Recently, a new category of tracking devices for swimming was introduced with the FORM Smart Swim Goggles (FORM Goggles). The goggles have a built-in see-through display and are capable of tracking and displaying distance, time splits, stroke, and pace metrics in real time using machine learning and augmented reality through a heads-up display. The purpose of this study was to assess the validity and reliability of the FORM Goggles compared with video analysis for stroke type, pool length count, pool length time, stroke rate, and stroke count in recreational swimmers and triathletes.

Method

A total of 36 participants performed mixed swimming intervals in a 25-m pool across two identical 900-m swim sessions performed at comparable intensities with 1 week interval. The participants wore FORM Goggles during their swims, which detected the following five swim metrics: stroke type, pool length time, pool length count, stroke count, and stroke rate. Four video cameras were positioned on the pool edges to capture ground truth video footage, which was then manually labeled by three trained individuals. Mean (SD) differences between FORM Goggles and ground truth were calculated for the selected metrics for both sessions. The absolute mean difference and mean absolute percentage error were used to assess the differences of the FORM Goggles relative to ground truth. The test-retest reliability of the goggles was assessed using both relative and absolute reliability metrics.

Results

Compared with video analysis, the FORM Goggles identified the correct stroke type at a rate of 99.7% (N = 2,354 pool lengths, p < 0.001), pool length count accuracy of 99.8%, and mean differences (FORM Goggles-ground truth) for pool length time: -0.10 s (1.49); stroke count: -0.63 (1.82); and stroke rate: 0.19 strokes/min (3.23). The test-retest intra-class correlation coefficient (ICC) values between the two test days were 0.793 for pool length time, 0.797 for stroke count, and 0.883 for stroke rate. Overall, for pool length time, the residuals were within ±1.0s for 65.3% of the total pool lengths, for stroke count within ±1 stroke for 62.6% of the total pool lengths, and for stroke rate within ±2 strokes/min for 66.40% of the total pool lengths.

Conclusion

The FORM Goggles were found valid and reliable for the tracking of pool length time, pool length count, stroke count, stroke rate, and stroke type during freestyle, backstroke, and breaststroke swimming in recreational swimmers and triathletes when compared with video analysis. This opens perspectives for receiving real-time information on performance metrics during swimming.

Artificial Intelligence-Based Opportunities in Liver Pathology-A Systematic Review

BACKGROUND

Artificial Intelligence (AI)-based Deep Neural Networks (DNNs) can handle a wide range of applications in image analysis, ranging from automated segmentation to diagnostic and prediction. As such, they have revolutionized healthcare, including in the liver pathology field.

OBJECTIVE

The present study aims to provide a systematic review of applications and performances provided by DNN algorithms in liver pathology throughout the Pubmed and Embase databases up to December 2022, for tumoral, metabolic and inflammatory fields.

RESULTS

42 articles were selected and fully reviewed. Each article was evaluated through the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool, highlighting their risks of bias.

CONCLUSIONS

DNN-based models are well represented in the field of liver pathology, and their applications are diverse. Most studies, however, presented at least one domain with a high risk of bias according to the QUADAS-2 tool. Hence, DNN models in liver pathology present future opportunities and persistent limitations. To our knowledge, this review is the first one solely focused on DNN-based applications in liver pathology, and to evaluate their bias through the lens of the QUADAS2 tool.

Where is laboratory medicine headed in the next decade? Partnership model for efficient integration and adoption of artificial intelligence into medical laboratories

OBJECTIVES

The field of artificial intelligence (AI) has grown in the past 10 years. Despite the crucial role of laboratory diagnostics in clinical decision-making, we found that the majority of AI studies focus on surgery, radiology, and oncology, and there is little attention given to AI integration into laboratory medicine.

METHODS

We dedicated a session at the 3rd annual European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) strategic conference in 2022 to the topic of AI in the laboratory of the future. The speakers collaborated on generating a concise summary of the content that is presented in this paper.

RESULTS

The five key messages are (1) Laboratory specialists and technicians will continue to improve the analytical portfolio, diagnostic quality and laboratory turnaround times; (2) The modularized nature of laboratory processes is amenable to AI solutions; (3) Laboratory sub-specialization continues and from test selection to interpretation, tasks increase in complexity; (4) Expertise in AI implementation and partnerships with industry will emerge as a professional competency and require novel educational strategies for broad implementation; and (5) regulatory frameworks and guidances have to be adopted to new computational paradigms.

CONCLUSIONS

In summary, the speakers opine that the ability to convert the value-proposition of AI in the laboratory will rely heavily on hands-on expertise and well designed quality improvement initiative from within laboratory for improved patient care.

Bootstrap Outlier Identification in Clinical Datasets for Lens Power Formula Constant Optimization

PURPOSE

Bootstrapping is a modern technique widely used in statistics to evaluate the performance of model parameters. The purpose of this study was to develop a strategy to identify and eliminate outliers in a dataset used for optimizing formula constants for lens power calculation.

METHODS

In a dataset with N = 888 clinical cases treated with a monofocal aspherical intraocular lens (XC1/XY1, Hoya) constants for the SRKT, Haigis and Castrop formula were optimized and the prediction error PE calculated. The PE was bootstrapped NB = 1000 times, and the mean and trimmed mean of the bootstrapped PE were derived to generate the Bootlier plot showing the probability density function of the mean minus trimmed mean. With outliers this Bootlier plot shows some multimodality, and a Bootlier Index was extracted as a measure for multimodality. Outliers were removed from the tails of the PE distribution in a stepwise fashion until the Bootlier Index fell below a threshold of 0.001.

RESULTS

With the entire dataset the mean/SD/median/mean absolute/root mean squared PE using the optimized formula constants were -0.0045/0.44415/0.0134/0.3406/0.4412 dpt with SRKT, 0.0065/0.3711/-0.0056/0.2830/0.3710 dpt with Haigis, and 0.0034/0.3452/0.0023/0.2683/0.3451 dpt with the Castrop formula. After identifying and removing outliers the respective metrics for the PE were -0.0036/0.4028/0.0134/0.3205/0.4026 dpt for the SRKT (13 cases removed), 0.0050/0.3375/-0.0056/0.2656/0.3373 dpt with Haigis (11 cases removed), and 0.0035/0.3168/0.0023/0.2531/0.3166 dpt with Castrop (11 cases removed). The multimodality in the Bootlier plots was reduced from 0/0.1567/0.0587/0.0258/0.0007/0 with SRKT, 0/0.0981/0.0261/0.0202/0.0003/0 with Haigis, and 0.0006/0.0006/0.0161/0.0191/0.0005/0 with Castrop for the entire dataset to values below 1e-3 for trimming both tails of the PE distribution by ⅛, ¼, ½, 1, 2.5, and 5% respectively.

CONCLUSION

We were able to prove that bootstrapping with outlier identification based on Bootlier plots and the Bootlier Index is a powerful tool to clean a dataset of outliers for formula constant optimization.

Machine Learning-Enabled NIR Spectroscopy in Assessing Powder Blend Uniformity: Clear-Up Disparities and Biases Induced by Physical Artefacts

NIR spectroscopy is a non-destructive characterization tool for the blend uniformity (BU) assessment. However, NIR spectra of powder blends often contain overlapping physical and chemical information of the samples. Deconvoluting the information related to chemical properties from that associated with the physical effects is one of the major objectives of this work. We achieve this aim in two ways. Firstly, we identified various sources of variability that might affect the BU results. Secondly, we leverage the machine learning-based sophisticated data analytics processes. To accomplish the aforementioned objectives, calibration samples of amlodipine as an active pharmaceutical ingredient (API) with the concentrations ranging between 67 and 133% w/w (dose ~ 3.6% w/w), in powder blends containing excipients, were prepared using a gravimetric approach and assessed using NIR spectroscopic analysis, followed by HPLC measurements. The bias in NIR results was investigated by employing data quality metrics (DQM) and bias-variance decomposition (BVD). To overcome the bias, the clustered regression (non-parametric and linear) was applied. We assessed the model's performance by employing the hold-out and k-fold internal cross-validation (CV). NIR-based blend homogeneity with low mean absolute error and an interval estimates of 0.674 (mean) ± 0.218 (standard deviation) w/w was established. Additionally, bootstrapping-based CV was leveraged as part of the NIR method lifecycle management that demonstrated the mean absolute error (MAE) of BU ± 3.5% w/w and BU ± 1.5% w/w for model generalizability and model transferability, respectively. A workflow integrating machine learning to NIR spectral analysis was established and implemented. Impact of various data learning approaches on NIR spectral data.

Evaluation of Robotic-Assisted Carotid Artery Stenting in a Virtual Model Using Motion-Based Performance Metrics

PURPOSE

Robotic-assisted carotid artery stenting (CAS) cases have been demonstrated with promising results. However, no quantitative measurements have been made to compare manual with robotic-assisted CAS. This study aims to quantify surgical performance using tool tip kinematic data and metrics of precision during CAS with manual and robotic control in an ex vivo model.

MATERIALS AND METHODS

Transfemoral CAS cases were performed in a high-fidelity endovascular simulator. Participants completed cases with manual and robotic techniques in 2 different carotid anatomies in random order. C-arm angulations, table position, and endovascular devices were standardized. Endovascular tool tip kinematic data were extracted. We calculated the spectral arc length (SPARC), average velocity, and idle time during navigation in the common carotid artery and lesion crossing. Procedural time, fluoroscopy time, movements of the deployed filter wire, precision of stent, and balloon positioning were recorded. Data were analyzed and compared between the 2 modalities.

RESULTS

Ten participants performed 40 CAS cases with a procedural success of 100% and 0% residual stenosis. The median procedural time was significantly higher during the robotic-assisted cases (seconds, median [interquartile range, IQR]: 128 [49.5] and 161.5 [62.5], p=0.02). Fluoroscopy time differed significantly between manual and robotic-assisted procedures (seconds, median [IQR]: 81.5 [32] and 98.5 [39.5], p=0.1). Movement of the deployed filter wire did not show significant difference between manual and robotic interventions (mm, median [IQR]: 13 [10.5] and 12.5 [11], p=0.5). The postdilation balloon exceeded the margin of the stent with a median of 2 [1] mm in both groups. Navigation with robotic assistance showed significantly lower SPARC values (-5.78±3.14 and -8.63±3.98, p=0.04) and higher idle time values (8.92±8.71 and 3.47±3.9, p=0.02) than those performed manually.

CONCLUSIONS

Robotic-assisted and manual CAS cases are comparable in the precision of stent and balloon positioning. Navigation in the carotid artery is associated with smoother motion and higher idle time values. These findings highlight the accuracy and the motion stabilizing capability of the endovascular robotic system.

CLINICAL IMPACT

Robotic assistance in the treatment of peripheral vascular disease is an emerging field and may be a tool for radiation protection and the geographic distribution of endovascular interventions in the future. This preclinical study compares the characteristics of manual and robotic-assisted carotid stenting (CAS). Our results highlight, that robotic-assisted CAS is associated with precise navigation and device positioning, and smoother navigation compared to manual CAS.

The Advantage of Case-Tailored Information Metrics for the Development of Predictive Models, Calculated Profit in Credit Scoring

This paper compares model development strategies based on different performance metrics. The study was conducted in the area of credit risk modeling with the usage of diverse metrics, including general-purpose Area Under the ROC curve (AUC), problem-dedicated Expected Maximum Profit (EMP) and the novel case-tailored Calculated Profit (CP). The metrics were used to optimize competitive credit risk scoring models based on two predictive algorithms that are widely used in the financial industry: Logistic Regression and extreme gradient boosting machine (XGBoost). A dataset provided by the American Fannie Mae agency was utilized to conduct the study. In addition to the baseline study, the paper also includes a stability analysis. In each case examined the proposed CP metric that allowed us to achieve the most profitable loan portfolio.

Analysis and modeling of COVID-19 epidemic dynamics in Saudi Arabia using SIR-PSO and machine learning approaches

INTRODUCTION

COVID-19 has become a global concern because it has extensive damage to health, social and economic systems worldwide. Consequently, there is an urgent need to develop tools to understand, analyze, monitor and control further outbreaks of the disease.

METHODOLOGY

The Susceptible Infected Recovered-Particle SwarmOptimization model and the feed-forward artificial neural network model were separately developed to model COVID-19 dynamics based on daily time-series data reported by the Saudi authorities from March 2, 2020 to February 21, 2021. The collected data were divided into training and validation datasets. The effectiveness of the investigated models was evaluated by using various performance metrics. The Susceptible-Infected-Recovered-Particle-Swarm-Optimization model was found to well predict the cumulative infected and recovered cases and to optimally tune the contact rate and the characteristic duration of the illness. The feed-forward artificial neural network model was found to be efficient in modeling daily new and cumulative infections, recoveries and deaths.

RESULTS

The forecasts provided by the investigated models had high coefficient of determination values of more than 0.97 and low mean absolute percentage errors (around 7% on average).

CONCLUSIONS

Both the Susceptible-Infected-Recovered-Particle-Swarm-Optimization and feed-forward artificial neural network models were efficient in modeling COVID-19 dynamics in Saudi Arabia. The results produced by the models can help the Saudi health authorities to analyze the virus dynamics and prepare efficient measures to control any future occurrence of the epidemic.

Design and Validation of Virtual Reality Task for Neuro-Rehabilitation of Distal Upper Extremities

Stroke, affecting approximately 15 million people worldwide, has long been a global cause of death and disability. Virtual Reality (VR) has shown its potential as an assistive tool for post-stroke rehabilitation. The objective of this pilot study was to define the task-specific performance metrics of VR tasks to assess the performance level of healthy subjects and patients quantitatively and to obtain their feedback for improving the developed framework. A pilot prospective study was designed. We tested the designed VR tasks on forty healthy right-handed subjects to evaluate its potential. Qualitative trajectory plots and three quantitative performance metrics—time taken to complete the task, percentage relative error, and trajectory smoothness—were computed from the recorded data of forty healthy subjects. Two patients with stroke were also enrolled to compare their performance with healthy subjects. Each participant received one VR session of 90 min. No adverse effects were noticed throughout the study. Performance metrics obtained from healthy subjects were used as a reference for patients. Relatively higher values of task completion time and trajectory smoothness and lower values of relative % error was observed for the affected hands w.r.t the unaffected hands of both the patients. For the unaffected hands of both the patients, the performance levels were found objectively closer to that of healthy subjects. A library of VR tasks for wrist and fingers were designed, and task-specific performance metrics were defined in this study. The evaluation of the VR exercises using these performance metrics will help the clinicians to assess the patient’s progress quantitatively and to design the rehabilitation framework for a future clinical study.

Online or Offline: Does It Matter? A Review of Existing Interpretation Approaches and Their Effect on Screening Mammography Metrics, Patient Satisfaction, and Cost

The ideal practice routine for screening mammography would optimize performance metrics and minimize costs, while also maximizing patient satisfaction. The main approaches to screening mammography interpretation include batch offline, non-batch offline, interrupted online, and uninterrupted online reading, each of which has its own advantages and drawbacks. This article reviews the current literature on approaches to screening mammography interpretation, potential effects of newer technologies, and promising artificial intelligence resources that could improve workflow efficiency in the future.

Use of surgical video-based automated performance metrics to predict blood loss and success of simulated vascular injury control in neurosurgery: a pilot study

OBJECTIVE

Experts can assess surgeon skill using surgical video, but a limited number of expert surgeons are available. Automated performance metrics (APMs) are a promising alternative but have not been created from operative videos in neurosurgery to date. The authors aimed to evaluate whether video-based APMs can predict task success and blood loss during endonasal endoscopic surgery in a validated cadaveric simulator of vascular injury of the internal carotid artery.

METHODS

Videos of cadaveric simulation trials by 73 neurosurgeons and otorhinolaryngologists were analyzed and manually annotated with bounding boxes to identify the surgical instruments in the frame. APMs in five domains were defined-instrument usage, time-to-phase, instrument disappearance, instrument movement, and instrument interactions-on the basis of expert analysis and task-specific surgical progressions. Bounding-box data of instrument position were then used to generate APMs for each trial. Multivariate linear regression was used to test for the associations between APMs and blood loss and task success (hemorrhage control in less than 5 minutes). The APMs of 93 successful trials were compared with the APMs of 49 unsuccessful trials.

RESULTS

In total, 29,151 frames of surgical video were annotated. Successful simulation trials had superior APMs in each domain, including proportionately more time spent with the key instruments in view (p < 0.001) and less time without hemorrhage control (p = 0.002). APMs in all domains improved in subsequent trials after the participants received personalized expert instruction. Attending surgeons had superior instrument usage, time-to-phase, and instrument disappearance metrics compared with resident surgeons (p < 0.01). APMs predicted surgeon performance better than surgeon training level or prior experience. A regression model that included APMs predicted blood loss with an R2 value of 0.87 (p < 0.001).

CONCLUSIONS

Video-based APMs were superior predictors of simulation trial success and blood loss than surgeon characteristics such as case volume and attending status. Surgeon educators can use APMs to assess competency, quantify performance, and provide actionable, structured feedback in order to improve patient outcomes. Validation of APMs provides a benchmark for further development of fully automated video assessment pipelines that utilize machine learning and computer vision.

Artificial Intelligence Algorithms in Visual Evoked Potential-Based Brain-Computer Interfaces for Motor Rehabilitation Applications: Systematic Review and Future Directions

Brain-Computer Interface (BCI) is a technology that uses electroencephalographic (EEG) signals to control external devices, such as Functional Electrical Stimulation (FES). Visual BCI paradigms based on P300 and Steady State Visually Evoked potentials (SSVEP) have shown high potential for clinical purposes. Numerous studies have been published on P300- and SSVEP-based non-invasive BCIs, but many of them present two shortcomings: (1) they are not aimed for motor rehabilitation applications, and (2) they do not report in detail the artificial intelligence (AI) methods used for classification, or their performance metrics. To address this gap, in this paper the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology was applied to prepare a systematic literature review (SLR). Papers older than 10 years, repeated or not related to a motor rehabilitation application, were excluded. Of all the studies, 51.02% referred to theoretical analysis of classification algorithms. Of the remaining, 28.48% were for spelling, 12.73% for diverse applications (control of wheelchair or home appliances), and only 7.77% were focused on motor rehabilitation. After the inclusion and exclusion criteria were applied and quality screening was performed, 34 articles were selected. Of them, 26.47% used the P300 and 55.8% the SSVEP signal. Five applications categories were established: Rehabilitation Systems (17.64%), Virtual Reality environments (23.52%), FES (17.64%), Orthosis (29.41%), and Prosthesis (11.76%). Of all the works, only four performed tests with patients. The most reported machine learning (ML) algorithms used for classification were linear discriminant analysis (LDA) (48.64%) and support vector machine (16.21%), while only one study used a deep learning algorithm: a Convolutional Neural Network (CNN). The reported accuracy ranged from 38.02 to 100%, and the Information Transfer Rate from 1.55 to 49.25 bits per minute. While LDA is still the most used AI algorithm, CNN has shown promising results, but due to their high technical implementation requirements, many researchers do not justify its implementation as worthwile. To achieve quick and accurate online BCIs for motor rehabilitation applications, future works on SSVEP-, P300-based and hybrid BCIs should focus on optimizing the visual stimulation module and the training stage of ML and DL algorithms.

From Forensics to Clinical Research: Expanding the Variant Calling Pipeline for the Precision ID mtDNA Whole Genome Panel

Despite a multitude of methods for the sample preparation, sequencing, and data analysis of mitochondrial DNA (mtDNA), the demand for innovation remains, particularly in comparison with nuclear DNA (nDNA) research. The Applied Biosystems™ Precision ID mtDNA Whole Genome Panel (Thermo Fisher Scientific, USA) is an innovative library preparation kit suitable for degraded samples and low DNA input. However, its bioinformatic processing occurs in the enterprise Ion Torrent Suite™ Software (TSS), yielding BAM files aligned to an unorthodox version of the revised Cambridge Reference Sequence (rCRS), with a heteroplasmy threshold level of 10%. Here, we present an alternative customizable pipeline, the PrecisionCallerPipeline (PCP), for processing samples with the correct rCRS output after Ion Torrent sequencing with the Precision ID library kit. Using 18 samples (3 original samples and 15 mixtures) derived from the 1000 Genomes Project, we achieved overall improved performance metrics in comparison with the proprietary TSS, with optimal performance at a 2.5% heteroplasmy threshold. We further validated our findings with 50 samples from an ongoing independent cohort of stroke patients, with PCP finding 98.31% of TSS's variants (TSS found 57.92% of PCP's variants), with a significant correlation between the variant levels of variants found with both pipelines.

Examining the Performance of Fog-Aided, Cloud-Centered IoT in a Real-World Environment

The fog layer provides substantial benefits in cloud-based IoT applications because it can serve as an aggregation layer and it moves the computation resources nearer to the IoT devices; however, it is important to ensure adequate performance is achieved in such applications, as the devices usually communicate frequently and authenticate with the cloud. This can cause performance and availability issues, which can be dangerous in critical applications such as in the healthcare sector. In this paper, we analyze the efficacy of the fog layer in different architectures in a real-world environment by examining performance metrics for the cloud and fog layers using different numbers of IoT devices. We also implement the fog layer using two methods to determine whether different fog implementation frameworks can affect the performance. The results show that including a fog layer with semi-heavyweight computation capability results in higher capital costs, although the in the long run resources, time, and money are saved. This study can serve as a reference for fundamental fog computing concepts. It can also be used to walk practitioners through different implementation frameworks of fog-aided IoT and to show tradeoffs in order to inform when to use each implementation framework based on one’s objectives.

Validation strategies for target prediction methods

Computational methods for target prediction, based on molecular similarity and network-based approaches, machine learning, docking and others, have evolved as valuable and powerful tools to aid the challenging task of mode of action identification for bioactive small molecules such as drugs and drug-like compounds. Critical to discerning the scope and limitations of a target prediction method is understanding how its performance was evaluated and reported. Ideally, large-scale prospective experiments are conducted to validate the performance of a model; however, this expensive and time-consuming endeavor is often not feasible. Therefore, to estimate the predictive power of a method, statistical validation based on retrospective knowledge is commonly used. There are multiple statistical validation techniques that vary in rigor. In this review we discuss the validation strategies employed, highlighting the usefulness and constraints of the validation schemes and metrics that are employed to measure and describe performance. We address the limitations of measuring only generalized performance, given that the underlying bioactivity and structural data are biased towards certain small-molecule scaffolds and target families, and suggest additional aspects of performance to consider in order to produce more detailed and realistic estimates of predictive power. Finally, we describe the validation strategies that were employed by some of the most thoroughly validated and accessible target prediction methods.

Design Considerations for Discrete Frequency Infrared Microscopy Systems

Discrete frequency infrared chemical imaging is transforming the practice of microspectroscopy by enabling a diversity of instrumentation and new measurement capabilities. While a variety of hardware implementations have been realized, design considerations that are unique to infrared (IR) microscopes have not yet been compiled in literature. Here, we describe the evolution of IR microscopes, provide rationales for design choices, and catalog some major considerations for each of the optical components in an imaging system. We analyze design choices that use these components to optimize performance, under their particular constraints, while providing illustrative examples. We then summarize a framework to assess the factors that determine an instrument's performance mathematically. Finally, we provide a validation approach by enumerating performance metrics that can be used to evaluate the capabilities of imaging systems or suitability for specific intended applications. Together, the presented concepts and examples should aid in understanding available instrument configurations, while guiding innovations in design of the next generation of IR chemical imaging spectrometers.

Developing, Validating and Implementing Performance Metrics to Evaluate the Health and Safety Performance of Sustainable Building Projects

IntroductionThis research aims to develop, validate and implement health and safety performance metrics to evaluate the health and safety performance of sustainable building projects throughout their design and construction in Manitoba.MethodsThirty-four metrics were developed following a detailed literature review and validated by expert judgment based on analytic soundness, practicality and predictability. Only 25 metrics satisfied these criteria, of which five metrics were implemented via data collected on seven sustainable buildings and seven non-sustainable ones.ResultsThe results showed that sustainable building projects had 12.7% higher recordable injuries rates than non-sustainable ones, although the difference was statistically insignificant. Findings from this research showed that for sustainable building and non-sustainable ones, PM19, "The percentage of workers with unsafe behaviour based on conducted safety observations" had a statistically significant and strong negative correlation with PM 8 "The percentage of workers who attended safety meetings" and with PM 21, "The percentage of the total workdays in which safety meeting were held".ConclusionThe findings of this research can be used by general contractors and safety practitioners looking to enact evidence-based guidance to manage safety proactively onsite and improve health and safety performance of their sustainable building projects.

Evaluating Performance of Microwave Image Reconstruction Algorithms: Extracting Tissue Types with Segmentation Using Machine Learning

Evaluating the quality of reconstructed images requires consistent approaches to extracting information and applying metrics. Partitioning medical images into tissue types permits the quantitative assessment of regions that contain a specific tissue. The assessment facilitates the evaluation of an imaging algorithm in terms of its ability to reconstruct the properties of various tissue types and identify anomalies. Microwave tomography is an imaging modality that is model-based and reconstructs an approximation of the actual internal spatial distribution of the dielectric properties of a breast over a reconstruction model consisting of discrete elements. The breast tissue types are characterized by their dielectric properties, so the complex permittivity profile that is reconstructed may be used to distinguish different tissue types. This manuscript presents a robust and flexible medical image segmentation technique to partition microwave breast images into tissue types in order to facilitate the evaluation of image quality. The approach combines an unsupervised machine learning method with statistical techniques. The key advantage for using the algorithm over other approaches, such as a threshold-based segmentation method, is that it supports this quantitative analysis without prior assumptions such as knowledge of the expected dielectric property values that characterize each tissue type. Moreover, it can be used for scenarios where there is a scarcity of data available for supervised learning. Microwave images are formed by solving an inverse scattering problem that is severely ill-posed, which has a significant impact on image quality. A number of strategies have been developed to alleviate the ill-posedness of the inverse scattering problem. The degree of success of each strategy varies, leading to reconstructions that have a wide range of image quality. A requirement for the segmentation technique is the ability to partition tissue types over a range of image qualities, which is demonstrated in the first part of the paper. The segmentation of images into regions of interest corresponding to various tissue types leads to the decomposition of the breast interior into disjoint tissue masks. An array of region and distance-based metrics are applied to compare masks extracted from reconstructed images and ground truth models. The quantitative results reveal the accuracy with which the geometric and dielectric properties are reconstructed. The incorporation of the segmentation that results in a framework that effectively furnishes the quantitative assessment of regions that contain a specific tissue is also demonstrated. The algorithm is applied to reconstructed microwave images derived from breasts with various densities and tissue distributions to demonstrate the flexibility of the algorithm and that it is not data-specific. The potential for using the algorithm to assist in diagnosis is exhibited with a tumor tracking example. This example also establishes the usefulness of the approach in evaluating the performance of the reconstruction algorithm in terms of its sensitivity and specificity to malignant tissue and its ability to accurately reconstruct malignant tissue.

A Survey of Smartphone-Based Indoor Positioning System Using RF-Based Wireless Technologies

In recent times, social and commercial interests in location-based services (LBS) are significantly increasing due to the rise in smart devices and technologies. The global navigation satellite systems (GNSS) have long been employed for LBS to navigate and determine accurate and reliable location information in outdoor environments. However, the GNSS signals are too weak to penetrate buildings and unable to provide reliable indoor LBS. Hence, GNSS’s incompetence in the indoor environment invites extensive research and development of an indoor positioning system (IPS). Various technologies and techniques have been studied for IPS development. This paper provides an overview of the available smartphone-based indoor localization solutions that rely on radio frequency technologies. As fingerprinting localization is mostly accepted for IPS development owing to its good localization accuracy, we discuss fingerprinting localization in detail. In particular, our analysis is more focused on practical IPS that are realized using a smartphone and Wi-Fi/Bluetooth Low Energy (BLE) as a signal source. Furthermore, we elaborate on the challenges of practical IPS, the available solutions and comprehensive performance comparison, and present some future trends in IPS development.

Classification of Aggressive Movements Using Smartwatches

Recognizing aggressive movements is a challenging task in human activity recognition. Wearable smartwatch technology with machine learning may be a viable approach for human aggressive behavior classification. This research identified a viable classification model and feature selector (CM-FS) combination for separating aggressive from non-aggressive movements using smartwatch data and determined if only one smartwatch is sufficient for this task. A ranking method was used to select relevant CM-FS models across accuracy, sensitivity, specificity, precision, F-score, and Matthews correlation coefficient (MCC). The Waikato environment for knowledge analysis (WEKA) was used to run 6 machine learning classifiers (random forest, k-nearest neighbors (kNN), multilayer perceptron neural network (MP), support vector machine, naïve Bayes, decision tree) coupled with three feature selectors (ReliefF, InfoGain, Correlation). Microsoft Band 2 accelerometer and gyroscope data were collected during an activity circuit that included aggressive (punching, shoving, slapping, shaking) and non-aggressive (clapping hands, waving, handshaking, opening/closing a door, typing on a keyboard) tasks. A combination of kNN and ReliefF was the best CM-FS model for separating aggressive actions from non-aggressive actions, with 99.6% accuracy, 98.4% sensitivity, 99.8% specificity, 98.9% precision, 0.987 F-score, and 0.984 MCC. kNN and random forest classifiers, combined with any of the feature selectors, generated the top models. Models with naïve Bayes or support vector machines had poor performance for sensitivity, F-score, and MCC. Wearing the smartwatch on the dominant wrist produced the best single-watch results. The kNN and ReliefF combination demonstrated that this smartwatch-based approach is a viable solution for identifying aggressive behavior. This wrist-based wearable sensor approach could be used by care providers in settings where people suffer from dementia or mental health disorders, where random aggressive behaviors often occur.

A Review of CFD Modelling and Performance Metrics for Osmotic Membrane Processes

Simulation via Computational Fluid Dynamics (CFD) offers a convenient way for visualising hydrodynamics and mass transport in spacer-filled membrane channels, facilitating further developments in spiral wound membrane (SWM) modules for desalination processes. This paper provides a review on the use of CFD modelling for the development of novel spacers used in the SWM modules for three types of osmotic membrane processes: reverse osmosis (RO), forward osmosis (FO) and pressure retarded osmosis (PRO). Currently, the modelling of mass transfer and fouling for complex spacer geometries is still limited. Compared with RO, CFD modelling for PRO is very rare owing to the relative infancy of this osmotically driven membrane process. Despite the rising popularity of multi-scale modelling of osmotic membrane processes, CFD can only be used for predicting process performance in the absence of fouling. This paper also reviews the most common metrics used for evaluating membrane module performance at the small and large scales.

A Deep Learning Framework for Assessing Physical Rehabilitation Exercises

Computer-aided assessment of physical rehabilitation entails evaluation of patient performance in completing prescribed rehabilitation exercises, based on processing movement data captured with a sensory system. Despite the essential role of rehabilitation assessment toward improved patient outcomes and reduced healthcare costs, existing approaches lack versatility, robustness, and practical relevance. In this paper, we propose a deep learning-based framework for automated assessment of the quality of physical rehabilitation exercises. The main components of the framework are metrics for quantifying movement performance, scoring functions for mapping the performance metrics into numerical scores of movement quality, and deep neural network models for generating quality scores of input movements via supervised learning. The proposed performance metric is defined based on the log-likelihood of a Gaussian mixture model, and encodes low-dimensional data representation obtained with a deep autoencoder network. The proposed deep spatio-temporal neural network arranges data into temporal pyramids, and exploits the spatial characteristics of human movements by using sub-networks to process joint displacements of individual body parts. The presented framework is validated using a dataset of ten rehabilitation exercises. The significance of this work is that it is the first that implements deep neural networks for assessment of rehabilitation performance.

Advancing Professionalization of Biobank Business Operations: Performance and Utilization

Biobanks are now in the spotlight as key enablers supporting preclinical, clinical, and environmental research. Awareness of their value has increased along with the need for these infrastructures to be sustained through business-focused practices. Following our 2017 pilot survey on biobank business planning, we initiated a more comprehensive 38-question multiple-language worldwide survey on biobank sustainability. Two hundred seventy-six biobanks of various sizes and stages of business planning (in place, in progress or none) responded. About two-thirds were established in the last 10 years. Survey results confirm our hypothesis that biobanks with business plans or preparing such plans are trending toward more professional structures. Specific survey data focusing on performance metrics and utilization, as related to sustainability, are presented. Biobanks most frequently measured basic performance metrics (sample utilization, samples collected, samples distributed, internal projects supported). Metrics less often reported included sample and data quality, cost recovery, citations, and publications, typically correlating with higher levels of biobank complexity and professionalism. Biobanks reported supporting projects for both internal and external use, with support of projects within their own organizations as the main driver of biobanks, independent of business plan status. Having a business plan seemed to be a key factor for biobanks that had developed sustained support for external commercial projects. While under half of the biobanks reported both target and actual utilization rates, the responses provided valuable data on utilization. Target utilization rates were much higher (2.5 to 5 times higher) than the rate of actual use. Many of the biobanks report less than 10% utilization. Biobanks with low utilization rates make sustainability a very distant and likely unreachable goal. Our survey has provided some basic data about biobank business planning globally. Continued research should be done, with the data and information shared within the community for the good of all biobank stakeholders.

Multi-State Energy Classifier to Evaluate the Performance of the NILM Algorithm

With the large-scale deployment of smart meters worldwide, research in non-intrusive load monitoring (NILM) has seen a significant rise due to its dual use of real-time monitoring of end-user appliances and user-centric feedback of power consumption usage. NILM is a technique for estimating the state and the power consumption of an individual appliance in a consumer's premise using a single point of measurement device such as a smart meter. Although there are several existing NILM techniques, there is no meaningful and accurate metric to evaluate these NILM techniques for multi-state devices such as the fridge, heat pump, etc. In this paper, we demonstrate the inadequacy of the existing metrics and propose a new metric that combines both event classification and energy estimation of an operational state to give a more realistic and accurate evaluation of the performance of the existing NILM techniques. In particular, we use unsupervised clustering techniques to identify the operational states of the device from a labeled dataset to compute a penalty threshold for predictions that are too far away from the ground truth. Our work includes experimental evaluation of the state-of-the-art NILM techniques on widely used datasets of power consumption data measured in a real-world environment.

A validation study of a virtual-based haptic system for endoscopic sinus surgery training

BACKGROUND

The development of endoscopic sinus surgery (ESS) training simulators for clinical environment applications has reduced the existing shortcomings in conventional teaching methods, creating a standard environment for trainers and trainees in a more accurate and repeatable fashion.

MATERIALS AND METHODS

In this research, the validation study of an ESS training simulator has been addressed. It is important to consider components that guide trainees to improve their hand movements control in the orbital floor removal in an ESS operation. Therefore, we defined three tasks to perform: pre-experiment learning, training, and evaluation. In these tasks, the critical regions introduced in the virtual training environment are forbidden to be touched. Recruiting 20 participants, divided into two groups, we investigated the performance metrics: quality (the percentage of the realism for the generated force for orbital floor removal and the usefulness of the proposed training system for the surgical educational curricula.), efficiency (time, path length), and safety (touching the goal and forbidden wall).

RESULTS

All recruited participants answered a post-evaluation questionnaire regarding their perceptions of training system realism, potential educational benefits, and practiced skills. We investigate the differences between groups' performance metrics by utilizing the analysis of variance-Kruskal-Wallis test. Acquired results indicate that training before the actual process of the surgery has a significant effect on the accuracy and validity of the process for surgeons.

CONCLUSIONS

Utilizing a standardized environment, trainers and trainees are able to carry out a process with regular features. In addition to traditional education methods, trainees can learn the risk of surgical operations. The training simulators can, also, provide a standard method for assessing the skills of surgical and medical students.

Multi-Level Comparison of Machine Learning Classifiers and Their Performance Metrics

Machine learning classification algorithms are widely used for the prediction and classification of the different properties of molecules such as toxicity or biological activity. the prediction of toxic vs. non-toxic molecules is important due to testing on living animals, which has ethical and cost drawbacks as well. The quality of classification models can be determined with several performance parameters. which often give conflicting results. In this study, we performed a multi-level comparison with the use of different performance metrics and machine learning classification methods. Well-established and standardized protocols for the machine learning tasks were used in each case. The comparison was applied to three datasets (acute and aquatic toxicities) and the robust, yet sensitive, sum of ranking differences (SRD) and analysis of variance (ANOVA) were applied for evaluation. The effect of dataset composition (balanced vs. imbalanced) and 2-class vs. multiclass classification scenarios was also studied. Most of the performance metrics are sensitive to dataset composition, especially in 2-class classification problems. The optimal machine learning algorithm also depends significantly on the composition of the dataset.

New Detection Paradigms to Improve Wireless Sensor Network Performance under Jamming Attacks

In this work, two new self-tuning collaborative-based mechanisms for jamming detection are proposed. These techniques are named (i) Connected Mechanism and (ii) Extended Mechanism. The first one detects jamming by comparing the performance parameters with respect to directly connected neighbors by interchanging packets with performance metric information, whereas the latter, jamming detection relays comparing defined zones of nodes related with a collector node, and using information of this collector detects a possible affected zone. The effectiveness of these techniques were tested in simulated environment of a quadrangular grid of 7 × 7, each node delivering 10 packets/sec, and defining as collector node, the one in the lower left corner of the grid. The jammer node is sending packets under reactive jamming. The mechanism was implemented and tested in AODV (Ad hoc On Demand Distance Vector), DSR (Dynamic Source Routing), and MPH (Multi-Parent Hierarchical), named AODV-M, DSR-M and MPH-M, respectively. Results reveal that the proposed techniques increase the accurate of the detected zone, reducing the detection of the affected zone up to 15% for AODV-M and DSR-M and up to 4% using the MPH-M protocol.

Commentary on "Measuring the Quality and Performance of Institutional Review Boards"

This commentary uses personal research ethics experience in South Africa to consider the position of Tsan who recommends a consortium to develop common metrics / performance indicators for determining quality of RECs / IRBs. Terms such as performance, quality and throughput need clarification in research metrics before effective metrics may be useful.

Application of the extreme scaling computing pattern on multiscale fusion plasma modelling

The extreme scaling pattern of the ComPat project is applied to a multi-scale workflow relevant to the magnetically confined fusion problem. This workflow combines transport, turbulence and equilibrium codes (together with additional auxiliaries such as initial conditions and numerical module), which aims at calculating the behaviour of a fusion plasma on long (transport) time scales based on information from much faster (turbulence) time scales. Initial findings of profile measurements are reported in this paper and indicate that, depending on the chosen performance metric for defining 'cost', such as time to completion, efficiency and total energy consumption of the mutliscale workflow, different choices on the number of cores would be made when determining the optimal execution configuration. A variant of the workflow which increases the inherent parallelism is presented, and shown to produce equivalent results at (typically) lower cost compared with the original workflow. This article is part of the theme issue 'Multiscale modelling, simulation and computing: from the desktop to the exascale'.

Evaluation of Silicon Wafer-Based Internal Reflection Elements for Use with in Situ Fourier Transform Infrared (FT-IR) Spectroscopy

Silicon wafer-based internal reflection elements (IREs) present many practical advantages over the prisms conventionally used for attenuated total reflection (ATR) spectroscopy in the infrared. We examine two methods of using minimally prepared IREs that have appeared in the literature, edge-coupled (EC) and prism-coupled (PC), in conjunction with a liquid flow cell. Polarization measurements show that radiation entering the PC-IRE becomes depolarized due to stress-induced birefringence, and transmission through the edge of the EC-IRE also affects the polarization state. Quantification of the noise and a calibration using a series of sodium acetate solutions show the sensitivity of the PC-IRE outweighs the lower noise obtainable with the EC-IRE.

Pay-for-Performance and Veteran Care in the VHA and the Community: a Systematic Review

BACKGROUND

Although pay-for-performance (P4P) strategies have been used by the Veterans Health Administration (VHA) for over a decade, the long-term benefits of P4P are unclear. The use of P4P is further complicated by the increased use of non-VHA healthcare providers as part of the Veterans Choice Program. We conducted a systematic review and key informant interviews to better understand the effectiveness and potential unintended consequences of P4P, as well as the implementation factors and design features important in both VHA and non-VHA/community settings.

METHODS

We searched PubMed, PsycINFO, and CINAHL through March 2017 and reviewed reference lists. We included trials and observational studies of P4P targeting Veteran health. Two investigators abstracted data and assessed study quality. We interviewed VHA stakeholders to gain further insight.

RESULTS

The literature search yielded 1031 titles and abstracts, of which 30 studies met pre-specified inclusion criteria. Twenty-five examined P4P in VHA settings and 5 in community settings. There was no strong evidence supporting the effectiveness of P4P in VHA settings. Interviews with 17 key informants were consistent with studies that identified the potential for overtreatment associated with performance metrics in the VHA. Key informants' views on P4P in community settings included the need to develop relationships with providers and health systems with records of strong performance, to improve coordination by targeting documentation and data sharing processes, and to troubleshoot the limited impact of P4P among practices where Veterans make up a small fraction of the patient population.

DISCUSSION

The evidence to support the effectiveness of P4P on Veteran health is limited. Key informants recognize the potential for unintended consequences, such as overtreatment in VHA settings, and suggest that implementation of P4P in the community focus on relationship building and target areas such as documentation and coordination of care.

Assessing the Quality and Performance of Human Research Protection Programs to Guide Compliance Oversight Activities

Routine on-site reviews should focus primarily on facilities that are at risk of harming human subjects. Using human research protection program performance metric data from 107 facilities, we defined a facility to be at risk when one of its noncompliance/incident rates was among the top three highest rates of that performance metric. Based on 14 performance metrics with noncompliance and incidents in 2017, 27 facilities were identified to be at risk. These 27 facilities at risk, while constituting only 25% of all facilities, contributed to 70% ± 25% ( M ± SD; range = 32%-100%) of all reported noncompliance/incidents. Thus, performance metric data can be used to guide compliance oversight activities.

Developing an objective assessment of surgical performance from operating room video and surgical imagery

An unbiased, repeatable process for assessing operating room performance is an important step toward quantifying the relationship between surgical training and performance. Hip fracture surgeries offer a promising first target in orthopedic trauma because they are common and they offer quantitative performance metrics that can be assessed from video recordings and intraoperative fluoroscopic images. Hip fracture repair surgeries were recorded using a head-mounted point-of-view camera. Intraoperative fluoroscopic images were also saved. The following performance metrics were analyzed: duration of wire navigation, number of fluoroscopic images collected, degree of intervention by the surgeon's supervisor, and the tip-apex distance (TAD). Two orthopedic traumatologists graded surgical performance in each video independently using an Objective Structured Assessment of Technical Skill (OSATS). Wire navigation duration correlated with weeks into residency and prior cases logged. TAD correlated with cases logged. There was no significant correlation between the OSATS total score and experience metrics. Total OSATS score correlated with duration and number of fluoroscopic images. Our results indicate that two metrics of hip fracture wire navigation performance, duration and TAD, significantly differentiate surgical experience. The methods presented have the potential to provide truly objective assessment of resident technical performance in the OR.

Integrating prior information into microwave tomography Part 1: Impact of detail on image quality

PURPOSE

The authors investigate the impact that incremental increases in the level of detail of patient-specific prior information have on image quality and the convergence behavior of an inversion algorithm in the context of near-field microwave breast imaging. A methodology is presented that uses image quality measures to characterize the ability of the algorithm to reconstruct both internal structures and lesions embedded in fibroglandular tissue. The approach permits key aspects that impact the quality of reconstruction of these structures to be identified and quantified. This provides insight into opportunities to improve image reconstruction performance.

METHODS

Patient-specific information is acquired using radar-based methods that form a regional map of the breast. This map is then incorporated into a microwave tomography algorithm. Previous investigations have demonstrated the effectiveness of this approach to improve image quality when applied to data generated with two-dimensional (2D) numerical models. The present study extends this work by generating prior information that is customized to vary the degree of structural detail to facilitate the investigation of the role of prior information in image formation. Numerical 2D breast models constructed from magnetic resonance (MR) scans, and reconstructions formed with a three-dimensional (3D) numerical breast model are used to assess if trends observed for the 2D results can be extended to 3D scenarios.

RESULTS

For the blind reconstruction scenario (i.e., no prior information), the breast surface is not accurately identified and internal structures are not clearly resolved. A substantial improvement in image quality is achieved by incorporating the skin surface map and constraining the imaging domain to the breast. Internal features within the breast appear in the reconstructed image. However, it is challenging to discriminate between adipose and glandular regions and there are inaccuracies in both the structural properties of the glandular region and the dielectric properties reconstructed within this structure. Using a regional map with a skin layer only marginally improves this situation. Increasing the structural detail in the prior information to include internal features leads to reconstructions for which the interface that delineates the fat and gland regions can be inferred. Different features within the glandular region corresponding to tissues with varying relative permittivity values, such as a lesion embedded within glandular structure, emerge in the reconstructed images.

CONCLUSION

Including knowledge of the breast surface and skin layer leads to a substantial improvement in image quality compared to the blind case, but the images have limited diagnostic utility for applications such as tumor response tracking. The diagnostic utility of the reconstruction technique is improved considerably when patient-specific structural information is used. This qualitative observation is supported quantitatively with image metrics.

Integrating prior information into microwave tomography part 2: Impact of errors in prior information on microwave tomography image quality

PURPOSE

The authors have developed a method to combine a patient-specific map of tissue structure and average dielectric properties with microwave tomography. The patient-specific map is acquired with radar-based techniques and serves as prior information for microwave tomography. The impact that the degree of structural detail included in this prior information has on image quality was reported in a previous investigation. The aim of the present study is to extend this previous work by identifying and quantifying the impact that errors in the prior information have on image quality, including the reconstruction of internal structures and lesions embedded in fibroglandular tissue. This study also extends the work of others reported in literature by emulating a clinical setting with a set of experiments that incorporate heterogeneity into both the breast interior and glandular region, as well as prior information related to both fat and glandular structures.

METHODS

Patient-specific structural information is acquired using radar-based methods that form a regional map of the breast. Errors are introduced to create a discrepancy in the geometry and electrical properties between the regional map and the model used to generate the data. This permits the impact that errors in the prior information have on image quality to be evaluated. Image quality is quantitatively assessed by measuring the ability of the algorithm to reconstruct both internal structures and lesions embedded in fibroglandular tissue. The study is conducted using both 2D and 3D numerical breast models constructed from MRI scans.

RESULTS

The reconstruction results demonstrate robustness of the method relative to errors in the dielectric properties of the background regional map, and to misalignment errors. These errors do not significantly influence the reconstruction accuracy of the underlying structures, or the ability of the algorithm to reconstruct malignant tissue. Although misalignment errors do not significantly impact the quality of the reconstructed fat and glandular structures for the 3D scenarios, the dielectric properties are reconstructed less accurately within the glandular structure for these cases relative to the 2D cases. However, general agreement between the 2D and 3D results was found.

CONCLUSION

A key contribution of this paper is the detailed analysis of the impact of prior information errors on the reconstruction accuracy and ability to detect tumors. The results support the utility of acquiring patient-specific information with radar-based techniques and incorporating this information into MWT. The method is robust to errors in the dielectric properties of the background regional map, and to misalignment errors. Completion of this analysis is an important step toward developing the method into a practical diagnostic tool.

Surface Estimation for Microwave Imaging

Biomedical imaging and sensing applications in many scenarios demand accurate surface estimation from a sparse set of noisy measurements. These measurements may arise from a variety of sensing modalities, including laser or electromagnetic samples of an object’s surface. We describe a state-of-the-art microwave imaging prototype that has sensors to acquire both microwave and laser measurements. The approach developed to translate sparse samples of the breast surface into an accurate estimate of the region of interest is detailed. To evaluate the efficacy of the method, laser and electromagnetic samples are acquired by sensors from three realistic breast models with varying sizes and shapes. A set of metrics is developed to assist with the investigation and demonstrate that the algorithm is able to accurately estimate the shape of a realistic breast phantom when only a sparse set of data are available. Moreover, the algorithm is robust to the presence of measurement noise, and is effective when applied to measurement scans of patients acquired with the prototype.

Selection of Fiducial Locations and Performance Metrics for Point-Based Rigid-Body Registration

A method is described to select the location and number of fiducials used in point-based, rigid-body registration of two coordinate frames. Two indices are introduced which are used to search for the optimum configuration of fiducials. They can be used to quickly evaluate a large number of configurations because no actual registration is involved in their calculation. Furthermore, configurations yielding small values of the indices correlate well with configurations which result in optimum registrations. Three registration performance metrics are discussed, and it is shown that optimization of different metrics leads to different selection of fiducial configurations. If an optimized configuration is selected as a starting configuration of N fiducials, the addition of extra fiducials does not significantly improve the registration in most cases. This work is based on 3D data acquired with three different instruments, each having different noise and bias characteristics.

Implementation Processes and Pay for Performance in Healthcare: A Systematic Review

BACKGROUND

Over the last decade, various pay-for-performance (P4P) programs have been implemented to improve quality in health systems, including the VHA. P4P programs are complex, and their effects may vary by design, context, and other implementation processes. We conducted a systematic review and key informant (KI) interviews to better understand the implementation factors that modify the effectiveness of P4P.

METHODS

We searched PubMed, PsycINFO, and CINAHL through April 2014, and reviewed reference lists. We included trials and observational studies of P4P implementation. Two investigators abstracted data and assessed study quality. We interviewed P4P researchers to gain further insight.

RESULTS

Among 1363 titles and abstracts, we selected 509 for full-text review, and included 41 primary studies. Of these 41 studies, 33 examined P4P programs in ambulatory settings, 7 targeted hospitals, and 1 study applied to nursing homes. Related to implementation, 13 studies examined program design, 8 examined implementation processes, 6 the outer setting, 18 the inner setting, and 5 provider characteristics. Results suggest the importance of considering underlying payment models and using statistically stringent methods of composite measure development, and ensuring that high-quality care will be maintained after incentive removal. We found no conclusive evidence that provider or practice characteristics relate to P4P effectiveness. Interviews with 14 KIs supported limited evidence that effective P4P program measures should be aligned with organizational goals, that incentive structures should be carefully considered, and that factors such as a strong infrastructure and public reporting may have a large influence.

DISCUSSION

There is limited evidence from which to draw firm conclusions related to P4P implementation. Findings from studies and KI interviews suggest that P4P programs should undergo regular evaluation and should target areas of poor performance. Additionally, measures and incentives should align with organizational priorities, and programs should allow for changes over time in response to data and provider input.

Best practices for evaluating single nucleotide variant calling methods for microbial genomics

Innovations in sequencing technologies have allowed biologists to make incredible advances in understanding biological systems. As experience grows, researchers increasingly recognize that analyzing the wealth of data provided by these new sequencing platforms requires careful attention to detail for robust results. Thus far, much of the scientific Communit’s focus for use in bacterial genomics has been on evaluating genome assembly algorithms and rigorously validating assembly program performance. Missing, however, is a focus on critical evaluation of variant callers for these genomes. Variant calling is essential for comparative genomics as it yields insights into nucleotide-level organismal differences. Variant calling is a multistep process with a host of potential error sources that may lead to incorrect variant calls. Identifying and resolving these incorrect calls is critical for bacterial genomics to advance. The goal of this review is to provide guidance on validating algorithms and pipelines used in variant calling for bacterial genomics. First, we will provide an overview of the variant calling procedures and the potential sources of error associated with the methods. We will then identify appropriate datasets for use in evaluating algorithms and describe statistical methods for evaluating algorithm performance. As variant calling moves from basic research to the applied setting, standardized methods for performance evaluation and reporting are required; it is our hope that this review provides the groundwork for the development of these standards.

Proficiency performance benchmarks for removal of simulated brain tumors using a virtual reality simulator NeuroTouch

OBJECTIVE

Assessment of neurosurgical technical skills involved in the resection of cerebral tumors in operative environments is complex. Educators emphasize the need to develop and use objective and meaningful assessment tools that are reliable and valid for assessing trainees' progress in acquiring surgical skills. The purpose of this study was to develop proficiency performance benchmarks for a newly proposed set of objective measures (metrics) of neurosurgical technical skills performance during simulated brain tumor resection using a new virtual reality simulator (NeuroTouch).

DESIGN

Each participant performed the resection of 18 simulated brain tumors of different complexity using the NeuroTouch platform. Surgical performance was computed using Tier 1 and Tier 2 metrics derived from NeuroTouch simulator data consisting of (1) safety metrics, including (a) volume of surrounding simulated normal brain tissue removed, (b) sum of forces utilized, and (c) maximum force applied during tumor resection; (2) quality of operation metric, which involved the percentage of tumor removed; and (3) efficiency metrics, including (a) instrument total tip path lengths and (b) frequency of pedal activation.

SETTING

All studies were conducted in the Neurosurgical Simulation Research Centre, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada.

PARTICIPANTS

A total of 33 participants were recruited, including 17 experts (board-certified neurosurgeons) and 16 novices (7 senior and 9 junior neurosurgery residents).

RESULTS

The results demonstrated that "expert" neurosurgeons resected less surrounding simulated normal brain tissue and less tumor tissue than residents. These data are consistent with the concept that "experts" focused more on safety of the surgical procedure compared with novices. By analyzing experts' neurosurgical technical skills performance on these different metrics, we were able to establish benchmarks for goal proficiency performance training of neurosurgery residents.

CONCLUSION

This study furthers our understanding of expert neurosurgical performance during the resection of simulated virtual reality tumors and provides neurosurgical trainees with predefined proficiency performance benchmarks designed to maximize the learning of specific surgical technical skills.