SubEpiPredict: A tutorial-based primer and toolbox for fitting and forecasting growth trajectories using the ensemble n-sub-epidemic modeling framework

An ensemble n-sub-epidemic modeling framework that integrates sub-epidemics to capture complex temporal dynamics has demonstrated powerful forecasting capability in previous works. This modeling framework can characterize complex epidemic patterns, including plateaus, epidemic resurgences, and epidemic waves characterized by multiple peaks of different sizes. In this tutorial paper, we introduce and illustrate SubEpiPredict, a user-friendly MATLAB toolbox for fitting and forecasting time series data using an ensemble n-sub-epidemic modeling framework. The toolbox can be used for model fitting, forecasting, and evaluation of model performance of the calibration and forecasting periods using metrics such as the weighted interval score (WIS). We also provide a detailed description of these methods including the concept of the n-sub-epidemic model, constructing ensemble forecasts from the top-ranking models, etc. For the illustration of the toolbox, we utilize publicly available daily COVID-19 death data at the national level for the United States. The MATLAB toolbox introduced in this paper can be very useful for a wider group of audiences, including policymakers, and can be easily utilized by those without extensive coding and modeling backgrounds.

Outcomes by time-to-OR for penetrating abdominal trauma patients

INTRODUCTION

Time-To-OR is a critical process measure for trauma performance. However, this measure has not consistently demonstrated improvement in outcome.

STUDY DESIGN

Using TQIP, we identified facilities by 75th percentile time-to-OR to categorize slow, average, and fast hospitals. Using a GEE model, we calculated odds of mortality for all penetrating abdominal trauma patients, firearm injuries only, and patients with major complication by facility speed. We additionally estimated odds of mortality at the patient level.

RESULTS

Odds of mortality for patients at slow facilities was 1.095; 95% CI: 0.746, 1.608; p = 0.64 compared to average. Fast facility OR = 0.941; 95% CI: 0.780, 1.133; p = 0.52. At the patient-level each additional minute of time-to-OR was associated with 1.5% decreased odds of in-hospital mortality (OR 0.985; 95% CI:0.981, 0.989; p < 0.001). For firearm-only patients, facility speed was not associated with odds of in-hospital mortality (p-value = 0.61). Person-level time-to-OR was associated with 1.8% decreased odds of in-hospital mortality (OR 0.982; 95% CI: 0.977, 0.987; p < 0.001) with each additional minute of time-to-OR. Similarly, failure-to-rescue analysis showed no difference in in-hospital mortality at the patient level (p = 0.62) and 0.4% decreased odds of in-hospital mortality with each additional minute of time-to-OR at the patient level (OR 0.996; 95% CI: 0.993, 0.999; p = 0.004).

CONCLUSION

Despite the use of time-to-OR as a metric of trauma performance, there is little evidence for improvement in mortality or complication rate with improved time-to-OR at the facility or patient level. Performance metrics for trauma should be developed that more appropriately approximate patient outcome.

Quantifying Interrater Agreement and Reliability Between Thoracic Pathologists: Paradoxical Behavior of Cohen's Kappa in the Presence of a High Prevalence of the Histopathologic Feature in Lung Cancer

Introduction

Cohen's kappa is often used to quantify the agreement between two pathologists. Nevertheless, a high prevalence of the feature of interest can lead to seemingly paradoxical results, such as low Cohen's kappa values despite high "observed agreement." Here, we investigate Cohen's kappa using data from histologic subtyping assessment of lung adenocarcinomas and introduce alternative measures that can overcome this "kappa paradox."

Methods

A total of 50 frozen sections from stage I lung adenocarcinomas less than or equal to 3 cm in size were independently reviewed by two pathologists to determine the absence or presence of five histologic patterns (lepidic, papillary, acinar, micropapillary, solid). For each pattern, observed agreement (proportion of cases with concordant "absent" or "present" ratings) and Cohen's kappa were calculated, along with Gwet's AC1.

Results

The prevalence of any amount of the histologic patterns ranged from 42% (solid) to 97% (acinar). On the basis of Cohen's kappa, there was substantial agreement for four of the five patterns (lepidic, 0.65; papillary, 0.67; micropapillary, 0.64; solid, 0.61). Acinar had the lowest Cohen's kappa (0.43, moderate agreement), despite having the highest observed agreement (88%). In contrast, Gwet's AC1 values were close to or higher than Cohen's kappa across patterns (lepidic, 0.64; papillary, 0.69; micropapillary, 0.71; solid, 0.73; acinar, 0.85). The proportion of positive versus negative agreement was 93% versus 50% for acinar.

Conclusions

Given the dependence of Cohen's kappa on feature prevalence, interrater agreement studies should include complementary indices such as Gwet's AC1 and proportions of specific agreement, especially in settings with a high prevalence of the feature of interest.

Farewell Springer… Hello Wiley : The tale of an academic scientific periodical -"20 years later" the Journal of Cell Communication and Signaling

Academic publishing is the support for dissemination of research findings that constitute the grounds upon which new orientations and improvements are based on sharing breaking ideas, critical analyses of data, and argumentations that sustain the development of collaborative research projects. The wide diffusion of new scientific findings is pivotal to the progress of medical sciences, a salient feature of human societal fullness and intellectual welfare. In a practical way, the value of academic publishing can be ascertained by its capacity to reach a wide number of readers from different fields that may provide the soil for interactive projects. The challenges are numerous (Zul in Challenges in Academic Publishing; Navigating the Obstacles, 2023). An examination of the means developed to survey the individual performances of scientists, based on their publications, has led me to comment in this editorial on pitfalls that muddle the way to upstanding evaluations mainly based on irrelevant metrics.

GastroNet: A Custom Deep Learning Approach for Classification of Anomalies in Gastrointestinal Endoscopy Images

INTRODUCTION

Among all cancer forms, gastrointestinal (GI) cancer is the most serious condition that spreads quickly and requires early detection. GI disorders claim the lives of up to nearly two million people worldwide. To lower the mortality rate from GI cancer, early detection is essential.

METHOD

For the identification of GI illnesses, such as polyps, stomach ulcers, and bleeding, endoscopy is the gold standard in the medical imaging industry. The numerous images produced by endoscopy require an enormous amount of time for the specialist to diagnose the disease. It makes manual diagnosis difficult and has sparked research on automatic computer-based approaches to diagnose all the generated images quickly and accurately. AI-based algorithms have already been used in endoscopy images with promising outcomes and have enhanced disease identification and classification with precision. However, there are still a lot of issues to be solved, including figuring out potential biases in algorithms and improving interpretability and generalizability.

RESULT

The proposed GastroNet model creates a system for classifying digestive problems for the Kvasir Version 1 dataset. The framework consists of different CNN layers with multiple filters, and average max-pooling is used to extract image features. The optimization of network parameters is done using the Stochastic Gradient Descent (SGD) algorithm.

CONCLUSION

Finally, the robustness of the proposed model is compared with other state-of-the-art models like VGG 19, ResNet 50, Inception, and Xception in terms of evaluation metrics.

Zinc-Bromine Rechargeable Batteries: From Device Configuration, Electrochemistry, Material to Performance Evaluation

Zinc-bromine rechargeable batteries (ZBRBs) are one of the most powerful candidates for next-generation energy storage due to their potentially lower material cost, deep discharge capability, non-flammable electrolytes, relatively long lifetime and good reversibility. However, many opportunities remain to improve the efficiency and stability of these batteries for long-life operation. Here, we discuss the device configurations, working mechanisms and performance evaluation of ZBRBs. Both non-flow (static) and flow-type cells are highlighted in detail in this review. The fundamental electrochemical aspects, including the key challenges and promising solutions, are discussed, with particular attention paid to zinc and bromine half-cells, as their performance plays a critical role in determining the electrochemical performance of the battery system. The following sections examine the key performance metrics of ZBRBs and assessment methods using various ex situ and in situ/operando techniques. The review concludes with insights into future developments and prospects for high-performance ZBRBs.

Class imbalance should not throw you off balance: Choosing the right classifiers and performance metrics for brain decoding with imbalanced data

Machine learning (ML) is increasingly used in cognitive, computational and clinical neuroscience. The reliable and efficient application of ML requires a sound understanding of its subtleties and limitations. Training ML models on datasets with imbalanced classes is a particularly common problem, and it can have severe consequences if not adequately addressed. With the neuroscience ML user in mind, this paper provides a didactic assessment of the class imbalance problem and illustrates its impact through systematic manipulation of data imbalance ratios in (i) simulated data and (ii) brain data recorded with electroencephalography (EEG), magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI). Our results illustrate how the widely-used Accuracy (Acc) metric, which measures the overall proportion of successful predictions, yields misleadingly high performances, as class imbalance increases. Because Acc weights the per-class ratios of correct predictions proportionally to class size, it largely disregards the performance on the minority class. A binary classification model that learns to systematically vote for the majority class will yield an artificially high decoding accuracy that directly reflects the imbalance between the two classes, rather than any genuine generalizable ability to discriminate between them. We show that other evaluation metrics such as the Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC), and the less common Balanced Accuracy (BAcc) metric - defined as the arithmetic mean between sensitivity and specificity, provide more reliable performance evaluations for imbalanced data. Our findings also highlight the robustness of Random Forest (RF), and the benefits of using stratified cross-validation and hyperprameter optimization to tackle data imbalance. Critically, for neuroscience ML applications that seek to minimize overall classification error, we recommend the routine use of BAcc, which in the specific case of balanced data is equivalent to using standard Acc, and readily extends to multi-class settings. Importantly, we present a list of recommendations for dealing with imbalanced data, as well as open-source code to allow the neuroscience community to replicate and extend our observations and explore alternative approaches to coping with imbalanced data.

Formal evaluation of construction safety performance metrics and a case for a balanced approach

INTRODUCTION

Measuring safety performance is crucial to making informed decisions that improve construction safety management. Traditional approaches to construction safety performance measurement primarily focus on injury and fatality rates, but researchers have recently proposed and tested alternative metrics such as safety leading indicators and safety climate assessments. Although researchers tend to extol the benefits of alternative metrics, they are studied in isolation and the potential weaknesses are rarely discussed, leaving a critical gap in knowledge.

METHOD

To address this limitation, this study aimed to evaluate existing safety performance against a set of pre-determined criteria and explore how multiple metrics may be used together to optimize strengths and offset weaknesses. For a well-rounded evaluation, the study included three evidence-based assessment criteria (i.e., the extent to which the metric is predictive, objective, and valid) and three subjective criteria (i.e., the extent to which each metric is clear, functional, and important). The evidence-based criteria were evaluated using a structured review of existing empirical evidence in literature, while the subjective criteria were evaluated using expert opinion solicited through the Delphi method.

RESULTS

The results revealed that no construction safety performance measurement metric is strong in all evaluation criteria, but many weaknesses may be addressed through research and development. It was also demonstrated that combining multiple complementary metrics may result in a more complete evaluation of the safety systems because multiple metrics offset respective strengths and weaknesses.

PRACTICAL APPLICATIONS

The study provides a holistic understanding of construction safety measurement that may guide safety professionals in their selection of metrics and assist researchers who seek more reliable dependent variables for intervention testing and safety performance trending.

Implementation of a Multidisciplinary Inflammatory Breast Cancer Program for Quality Improvement

BACKGROUND

Inflammatory Breast Cancer (IBC) is a rare but aggressive subtype of breast cancer accounting for only 1% to 5% of cases but comprising 7% to 10% of breast cancer deaths. Diagnosis of IBC can be challenging which can lead to delays in diagnosis and treatment. We formed a multidisciplinary IBC program to address the unique challenges of diagnosing and treating patients with IBC.

MATERIALS AND METHODS

We retrospectively identified patients with an IBC CPT code and collected data on the date of the first visit with medical oncology, surgical oncology, or radiation oncology, date of biopsy, and initiation of neoadjuvant chemotherapy. In 2020, as part of the IBC program at The Ohio State University, the decision tree (DT) was revised to help identify potential IBC patients. These patients were prioritized with a multidisciplinary appointment within 3 days.

RESULTS

After adjusting the call center DT, there was a significant decline in the median and mean time from initial contact to chemotherapy initiation and an insignificant decrease in the mean time from contact to biopsy (P = .71884). The median time of contact to chemotherapy was 10 days (range 9-14) in 2020, a decrease of 43% compared to 3 prior years (P = .0068). After initiation of the IBC program, 100% of patients underwent trimodality therapy-neoadjuvant systemic therapy, modified radical mastectomy, and post mastectomy radiation therapy.

CONCLUSION

A multidisciplinary IBC program that included scheduling DT with specific questions about IBC symptoms helped identify potential patients and significantly decrease time to treatment and assured completion of trimodality therapy.

Suitability of EPDs for Supporting Life Cycle and Comparative Analysis of Concrete Mixtures

The use of environmental product declarations (EPDs) of concrete and other construction materials is gaining momentum. EPDs should enable an informed selection of products with a lower environmental footprint; hence, the issue of EPD comparability is highly relevant. In this paper, we identified and discussed the present shortcomings and future opportunities that can promote a meaningful EPD comparison for concrete products. Based on the published EPDs, we suggest a more comprehensive water consumption accounting, as the batching water is commonly underestimated. A set of performance metrics required to be specified for concrete are proposed to be included in the product category rules. An effort to develop a procedure for the regular calibration of existing tools with identical calculation databases and methods can produce outcomes that differ by 1-19%. The incorporation of prescriptive and consistently implemented life cycle inventory can minimize the calculation noise. The incorporation of uncertainty and variability as well as a supply-chain-specific EPD creation can help move toward a robust comparison based on the existing data in EPDs.

An optimal and efficient data security technique through crypto-stegano for E-commerce

E-Commerce or Electronic commerce is the buying and selling of goods and services in which any commercial transactions through wireless electronic devices such as hand-held computers (tablets), mobile phones or laptops is conducted anytime & anywhere via Internet technology. But, E-Commerce transactions or services are suffered by many attacks such as Man in the Middle attack, eavesdropping attacks, and etc. due to the lack of secured security infrastructure. Here, data security is one of the ways to keep the confidential information secure through E-Commerce transactions. In this connection, we have proposed an optimal and efficient data security with the combination of Elgamal cryptosystem and LSB image steganography technique for E-Commerce. In our proposed work, at the merchant side, Elgamal encryption technique is used to protect sensitive information during E-Commerce transactions from intruders and LSB image steganography process is also applied to hide generated Elgamal encrypted data and produce a stego-image (steganography image). Then, DCT (Discrete Cosine Transform) technique through autoencoder is imposed on stego-image to make an optimal image to increase the throughput of the work. After that, the produced optimal image with cipher text is sent to the customer end. At the customer end, first, stego-image is extracted from the optimal image using LSB retrieval process. Then, Elgamal decryption process is used to retrieve the original data and secure the E-Commerce transactions in efficiently. Based on the experiment, we have plotted the performance metrics such as MSE, PSNR and SSIM on the work and entropy of the optimal image is also calculated with respect to the sample image. Thereby, a high level performance metrics is obtained in our proposed work.

Forecasting commodity prices: empirical evidence using deep learning tools

Since the last two decades, financial markets have exhibited several transformations owing to recurring crises episodes that has led to the development of alternative assets. Particularly, the commodity market has attracted attention from investors and hedgers. However, the operational research stream has also developed substantially based on the growth of the artificial intelligence field, which includes machine learning and deep learning. The choice of algorithms in both machine learning and deep learning is case-sensitive. Hence, AI practitioners should first attempt solutions related to machine learning algorithms, and if such solutions are unsatisfactory, they must apply deep learning algorithms. Using this perspective, this study aims to investigate the potential of various deep learning basic algorithms for forecasting selected commodity prices. Formally, we use the Bloomberg Commodity Index (noted by the Global Aggregate Index) and its five component indices: Bloomberg Agriculture Subindex, Bloomberg Precious Metals Subindex, Bloomberg Livestock Subindex, Bloomberg Industrial Metals Subindex, and Bloomberg Energy Subindex. Based on daily data from January 2002 (the beginning wave of commodity markets' financialization) to December 2020, results show the effectiveness of the Long Short-Term Memory method as a forecasting tool and the superiority of the Bloomberg Livestock Subindex and Bloomberg Industrial Metals Subindex for assessing other commodities' indices. These findings is important in term for investors in term of risk management as well as policymakers in adjusting public policy, especially during Russian-Ukrainian war.

PToPI: A Comprehensive Review, Analysis, and Knowledge Representation of Binary Classification Performance Measures/Metrics

Although few performance evaluation instruments have been used conventionally in different machine learning-based classification problem domains, there are numerous ones defined in the literature. This study reviews and describes performance instruments via formally defined novel concepts and clarifies the terminology. The study first highlights the issues in performance evaluation via a survey of 78 mobile-malware classification studies and reviews terminology. Based on three research questions, it proposes novel concepts to identify characteristics, similarities, and differences of instruments that are categorized into 'performance measures' and 'performance metrics' in the classification context for the first time. The concepts reflecting the intrinsic properties of instruments such as canonical form, geometry, duality, complementation, dependency, and leveling, aim to reveal similarities and differences of numerous instruments, such as redundancy and ground-truth versus prediction focuses. As an application of knowledge representation, we introduced a new exploratory table called PToPI (Periodic Table of Performance Instruments) for 29 measures and 28 metrics (69 instruments including variant and parametric ones). Visualizing proposed concepts, PToPI provides a new relational structure for the instruments including graphical, probabilistic, and entropic ones to see their properties and dependencies all in one place. Applications of the exploratory table in six examples from different domains in the literature have shown that PToPI aids overall instrument analysis and selection of the proper performance metrics according to the specific requirements of a classification problem. We expect that the proposed concepts and PToPI will help researchers comprehend and use the instruments and follow a systematic approach to classification performance evaluation and publication.

A systematic literature review of cyber-security data repositories and performance assessment metrics for semi-supervised learning

In Machine Learning, the datasets used to build models are one of the main factors limiting what these models can achieve and how good their predictive performance is. Machine Learning applications for cyber-security or computer security are numerous including cyber threat mitigation and security infrastructure enhancement through pattern recognition, real-time attack detection, and in-depth penetration testing. Therefore, for these applications in particular, the datasets used to build the models must be carefully thought to be representative of real-world data. However, because of the scarcity of labelled data and the cost of manually labelling positive examples, there is a growing corpus of literature utilizing Semi-Supervised Learning with cyber-security data repositories. In this work, we provide a comprehensive overview of publicly available data repositories and datasets used for building computer security or cyber-security systems based on Semi-Supervised Learning, where only a few labels are necessary or available for building strong models. We highlight the strengths and limitations of the data repositories and sets and provide an analysis of the performance assessment metrics used to evaluate the built models. Finally, we discuss open challenges and provide future research directions for using cyber-security datasets and evaluating models built upon them.

Machine Learning Using Neural Networks for Metabolomic Pathway Analyses

Elucidating the mechanisms of metabolic pathways helps us understand the cascade of enzyme-catalyzed reactions that lead to the conversion of substances into final products. This has implications for predicting how newly synthesized compounds will affect a person's metabolism and, hence, the development of novel treatments to improve one's health. The study of metabolomic pathways, together with protein engineering, may also aid in the extraction, at a scale, of natural products to be used as drugs and drug precursors. Several approaches have been used to correlate protein annotations to metabolic pathways in order to derive pathways directly related to specific organisms. These could range from association rule-mining techniques to machine learning methods such as decision trees, naïve Bayes, logistic regression, and ensemble methods.In this chapter, we will be reviewing the use of machine learning for metabolic pathway analyses, with a step-by-step focus on the use of deep learning to predict the association of compounds (metabolites) to their respective metabolomic pathway classes. This prediction could help explain interactions of small molecules in organisms. Inspired by the work of Baranwal et al. (2019), we demonstrate how to build and train a deep learning neural network model to perform a multi-label prediction. We considered two different types of fingerprints as features (inputs to the model). The output of the model is the set of metabolic pathway classes (from the KEGG dataset) in which the input molecule participates. We will walk through the various steps of this process, including data collection, feature engineering, model selection, training, and evaluation. This model-building and evaluation process may be easily transferred to other domains of interest. All the source code used in this chapter is made publicly available at https://github.com/jp-um/machine_learning_for_metabolomic_pathway_analyses .

An Overview on Security and Privacy of Data in IoMT Devices: Performance Metrics, Merits, Demerits, and Challenges

The Internet of Medical Things (IoMT) emerges with new trendsetter device applications, where it defines the incorporation of medical devices with the Internet of Things (IoT). The healthcare sector continues to encounter challenging obstacles that have an impact on the quality of treatment provided to patients. To get rid of this problem, IoMT is being deployed to achieve the high reliability and efficiency of the health system. The IoMT devices are superimposed with clinical information as they contain the details of patient health data, address, and other patient identifiers. By containing such amount of sensitive information, it becomes cumbersome to preserve data privacy and security. Due to inadequate security and privacy precautions, patient health data is susceptible to leakage, which has a direct impact on the patient's life. In addition, the majority of medical devices are susceptible to cyberattacks, putting patient information at risk. Inadequate control of life-support equipment can have a devastating effect on patient outcomes. Thus, this survey intends to review the various security models of IoMT devices using standard techniques to support health care systems. It provides a wide range of literature reviews regarding IoMT systems and compares them with traditional methodologies. This review work exhibits the motivation for current technologies to maintain the security and privacy of patients' data with IoMT devices. The systematic review entails background on security in IoMT devices, techniques for security, usage of diverse validation measures, and also discusses the problems and motivation for future research work.

Robot-assisted ex vivo neobladder reconstruction: preliminary results of surgical skill evaluation

PURPOSE

Advanced developments in the medical field have gradually increased the public demand for surgical skill evaluation. However, this assessment always depends on the direct observation of experienced surgeons, which is time-consuming and variable. The introduction of robot-assisted surgery provides a new possibility for this evaluation paradigm. This paper aims at evaluating surgeon performance automatically with novel evaluation metrics based on different surgical data.

METHODS

Urologists ([Formula: see text]) from a hospital were requested to perform a simplified neobladder reconstruction on an ex vivo setup twice with different camera modalities ([Formula: see text]) randomly. They were divided into novices and experts ([Formula: see text], respectively) according to their experience in robot-assisted surgeries. Different performance metrics ([Formula: see text]) are proposed to achieve the surgical skill evaluation, considering both instruments and endoscope. Also, nonparametric tests are adopted to check if there are significant differences when evaluating surgeons performance.

RESULTS

When grouping according to four stages of neobladder reconstruction, statistically significant differences can be appreciated in phase 1 ([Formula: see text]) and phase 2 ([Formula: see text]) with normalized time-related metrics and camera movement-related metrics, respectively. On the other hand, considering experience grouping shows that both metrics are able to highlight statistically significant differences between novice and expert performances in the control protocol. It also shows that the camera-related performance of experts is significantly different ([Formula: see text]) when handling the endoscope manually and when it is automatic.

CONCLUSION

Surgical skill evaluation, using the approach in this paper, can effectively measure surgical procedures of surgeons with different experience. Preliminary results demonstrate that different surgical data can be fully utilized to improve the reliability of surgical evaluation. It also demonstrates its versatility and potential in the quantitative assessment of various surgical operations.

Reply to STOTEN 802 (2022) 149713: The fallacy in the use of the "best-fit" solution in hydrologic modeling

In this reply we respond to the short discussion contribution by Abbaspour (2022) in which a fallacy in the use of "best-fit" model solutions to be employed in hydrologic modeling studies is illustrated. Abbaspour (2022) advised to perform stochastic model calibration and proposed to employ the R- and P-Factor statistics for a model evaluation together with suggested thresholds for a model to be acceptable. In a minimal working example we followed the proposed stochastic approach for model evaluation and show that the proposed R- and P-Factor metrics and their thresholds accept implausible model ensemble simulations which would have been rejected in an individual assessment with the NSE metric. In this way, we want to raise the caution to rely on single performance metrics for model evaluation and the use of globally defined thresholds to define model acceptance.

Deep learning approaches for detecting DDoS attacks: a systematic review

In today's world, technology has become an inevitable part of human life. In fact, during the Covid-19 pandemic, everything from the corporate world to educational institutes has shifted from offline to online. It leads to exponential increase in intrusions and attacks over the Internet-based technologies. One of the lethal threat surfacing is the Distributed Denial of Service (DDoS) attack that can cripple down Internet-based services and applications in no time. The attackers are updating their skill strategies continuously and hence elude the existing detection mechanisms. Since the volume of data generated and stored has increased manifolds, the traditional detection mechanisms are not appropriate for detecting novel DDoS attacks. This paper systematically reviews the prominent literature specifically in deep learning to detect DDoS. The authors have explored four extensively used digital libraries (IEEE, ACM, ScienceDirect, Springer) and one scholarly search engine (Google scholar) for searching the recent literature. We have analyzed the relevant studies and the results of the SLR are categorized into five main research areas: (i) the different types of DDoS attack detection deep learning approaches, (ii) the methodologies, strengths, and weaknesses of existing deep learning approaches for DDoS attacks detection (iii) benchmarked datasets and classes of attacks in datasets used in the existing literature, and (iv) the preprocessing strategies, hyperparameter values, experimental setups, and performance metrics used in the existing literature (v) the research gaps, and future directions.

Evaluating the performance of drug-repurposing technologies

Drug-repurposing technologies are growing in number and maturing. However, comparisons to each other and to reality are hindered because of a lack of consensus with respect to performance evaluation. Such comparability is necessary to determine scientific merit and to ensure that only meaningful predictions from repurposing technologies carry through to further validation and eventual patient use. Here, we review and compare performance evaluation measures for these technologies using version 2 of our shotgun repurposing Computational Analysis of Novel Drug Opportunities (CANDO) platform to illustrate their benefits, drawbacks, and limitations. Understanding and using different performance evaluation metrics ensures robust cross-platform comparability, enabling us to continue to strive toward optimal repurposing by decreasing the time and cost of drug discovery and development.

Predicting the consequences of accidents involving dangerous substances using machine learning

A new dimension of learning lessons from the occurrence of hazardous events involving dangerous substances is considered relying on the availability of representative data and the significant evolution of a wide range of machine learning tools. The importance of such a dimension lies in the possibility of predicting the associated nature of damages without imposing any unrealistic simplifications or restrictions. To provide the best possible modeling framework, several implementations are tested using logistic regression, decision trees, neural networks, support vector machine, naive Bayes classifier and random forests to forecast the occurrence of the human, environmental and material consequences of industrial accidents based on the EU Major Accident Reporting System's records. Many performance metrics are estimated to select the most suitable model in each treated case. The obtained results show the distinctive ability of random forests and neural networks to predict the occurrence of specific consequences of accidents in the industrial installations, with an obvious exception concerning the performance of this latter algorithm when the involved datasets are highly unbalanced.

Quantifying the performance of dual-use rainwater harvesting systems

Rainwater harvesting systems in urban settings are increasingly relied upon to mitigate pluvial flooding on top of providing an additional water supply. Alternative designs have been proposed to support their dual use. Stormwater management performance is typically evaluated through long-term averages. However, long-term assessment is not aligned with the goal of attenuating the impacts of short duration high-intensity rainfall events. This paper contributes a framework for evaluating the dual-use performance of design alternatives. The framework incorporates a set of stormwater management metrics that provides a robust characterisation of performance during significant rainfall events. To the usual long-term volumetric retention metric, we add: 1) metrics that represent the total volume and duration above predevelopment (greenfield) runoff rates; and 2) robust peak outflow rate and retention efficiencies based on the long-term median of a representative sample of significant rainfall events. Our multi-criteria performance visualisations of alternative dual-use designs highlight the importance of carefully designing the forecast-based controlled release mechanisms built into active systems. This work has direct implications for design guidance standards, which we discuss.

An Approach to Improving Clinical Morale: Targeting a Positive Culture

Establishing a positive work environment is critical for successful outcomes and optimized performance. In a high-stakes arena such as healthcare, having all individuals engaged and ready to be part of the team is crucial. In this paper, we highlight a novel strategy for establishing effective communication and breaking down barriers. Through the implementation of a positive culture committee the neurosurgical department will improve relationships and performance. Goals and different strategies are addressed in the discussion. As implementation is underway, we plan to measure improved performance metrics in the coming years.

Differences Between Center-level Outcomes in Emergency and Elective General Surgery

BACKGROUND

Center-level outcome metrics have long been tracked in elective surgery (ELS). Despite recent interest in measuring emergency general surgery (EGS) quality, centers are often compared based on elective or combined outcomes. Therefore, quality of care for emergency surgery specifically is unknown.

METHODS

We extracted data on EGS and ELS patients from the 2016 State Inpatient Databases of Florida, New York, and Kentucky. Centers that performed >100 ELS and EGS operations were included. Risk-adjusted mortality, complication, and failure to rescue (FTR, death after complication) rates were calculated and observed-to-expected ratios were calculated by center for ELS and EGS patients. Centers were determined to be high or low outliers if the 90% CI for the observed: expected ratio excluded 1. We calculated the frequency with which centers demonstrated a different performance status between EGS and ELS. Kendall's tau values were calculated to assess for correlation between EGS and ELS status.

RESULTS

A total of 204 centers with 45,500 EGS cases and 49,380 ELS cases met inclusion criteria. Overall mortality, complication, and FTR rates were 1.7%, 8.0%, and 14.5% respectively. There was no significant correlation between mortality performance in EGS and ELS, with 36 centers in a different performance category (high outlier, low outlier, as expected) in EGS than in ELS. The correlation for complication rates was 0.20, with 60 centers in different categories for EGS and ELS. For FTR rates, there was no correlation, with 16 centers changing category.

CONCLUSIONS

There was minimal correlation between outcomes for ELS and EGS. High performers in one category were rarely high performers in the other. There may be important differences between the processes of care that are important for EGS and ELS outcomes that may yield meaningful opportunities for quality improvement.

Effects of airflow on the thermal environment and energy efficiency in raised-floor data centers: A review

Airflow is crucial for air-cooled data centers. Its flow path and distribution influences the thermal environment and energy efficiency of raised-floor data centers. This paper provides a review of the topic including the aspects of airflow factors, numerical study, airflow performance metrics, and thermal optimization. Based on the multi-scale characteristics of the data center, the thermal environment is categorized into room-level, rack-level, and server-level environments. For the room-level thermal environment, the main factors include layout, raised floor plenum and ceiling height, and perforated tiles. For the rack level, the effects of the porosity ratio of rack door, airflow rate/temperature, server population, server arrangement and power density are considered. For the server level, airflow rate and server fan speed are investigated. Moreover, numerical studies have been widely employed to understand the thermal environment of data centers. The selections of simulation tool and the methods for simplifying and validating the models are key to predicting the data center's thermal behavior correctly. In addition, airflow performance metrics and multi-scale thermal optimization are summarized and discussed. This review aims to emphasize the importance of the airflow in data centers and thus serve a guiding reference for airflow design and energy efficiency in data centers. Some recommended topics for future research are also provided.

Data of vertical and horizontal handover on video transmission in Proxy Mobile IPv6

The Internet Engineering Task Force provides a network-based mobility management solution to execute handover in heterogeneous networks on network-side called Proxy Mobile IPv6 (PMIPv6). In this data article, data are presented during the horizontal and vertical handover on video communication in PMIPv6 mobility protocols. The handover data are gathered using several measurement factors, which are latency, jitter, cumulative measured, and peak signal noise ratio under network simulation software, for both horizontal and vertical handovers [8].

Prediction of human drug-induced liver injury (DILI) in relation to oral doses and blood concentrations

Drug-induced liver injury (DILI) cannot be accurately predicted by animal models. In addition, currently available in vitro methods do not allow for the estimation of hepatotoxic doses or the determination of an acceptable daily intake (ADI). To overcome this limitation, an in vitro/in silico method was established that predicts the risk of human DILI in relation to oral doses and blood concentrations. This method can be used to estimate DILI risk if the maximal blood concentration (C_max) of the test compound is known. Moreover, an ADI can be estimated even for compounds without information on blood concentrations. To systematically optimize the in vitro system, two novel test performance metrics were introduced, the toxicity separation index (TSI) which quantifies how well a test differentiates between hepatotoxic and non-hepatotoxic compounds, and the toxicity estimation index (TEI) which measures how well hepatotoxic blood concentrations in vivo can be estimated. In vitro test performance was optimized for a training set of 28 compounds, based on TSI and TEI, demonstrating that (1) concentrations where cytotoxicity first becomes evident in vitro (EC₁₀) yielded better metrics than higher toxicity thresholds (EC₅₀); (2) compound incubation for 48 h was better than 24 h, with no further improvement of TSI after 7 days incubation; (3) metrics were moderately improved by adding gene expression to the test battery; (4) evaluation of pharmacokinetic parameters demonstrated that total blood compound concentrations and the 95%-population-based percentile of C_max were best suited to estimate human toxicity. With a support vector machine-based classifier, using EC₁₀ and C_max as variables, the cross-validated sensitivity, specificity and accuracy for hepatotoxicity prediction were 100, 88 and 93%, respectively. Concentrations in the culture medium allowed extrapolation to blood concentrations in vivo that are associated with a specific probability of hepatotoxicity and the corresponding oral doses were obtained by reverse modeling. Application of this in vitro/in silico method to the rat hepatotoxicant pulegone resulted in an ADI that was similar to values previously established based on animal experiments. In conclusion, the proposed method links oral doses and blood concentrations of test compounds to the probability of hepatotoxicity.

Assessing taxonomic metagenome profilers with OPAL

The explosive growth in taxonomic metagenome profiling methods over the past years has created a need for systematic comparisons using relevant performance criteria. The Open-community Profiling Assessment tooL (OPAL) implements commonly used performance metrics, including those of the first challenge of the initiative for the Critical Assessment of Metagenome Interpretation (CAMI), together with convenient visualizations. In addition, we perform in-depth performance comparisons with seven profilers on datasets of CAMI and the Human Microbiome Project. OPAL is freely available at https://github.com/CAMI-challenge/OPAL .

LS-GSNO and CWSNO Enhancement Processes Using PCA Algorithm with LOOCV of R-SM Technique for Effective Face Recognition Approach

The eminence of image under test is identified with different methods of Face Recognition (FR) which results in failure due to rapid change in pixel intensity. The identification of similar face with inter class similarity is very difficult in imaging. The imaging technology faces difficult in the mounting of intra class variability because of accommodate, intra-class variability because of head pose, illumination conditions, expressions, facial accessories, aging effects and cartoon faces. In the earliest approach, gradient with Zernike momemts were used to regonize the faces, the performance is low to overcome this a new approach is introduced. Many features of FR are affected by the outcome and low occurrence of performance is observed which is applicable only for data sets that are smaller. The introduction of a new approach can overcome the above stated limitations. This paper describes a novel approach for LS enhancement technique using GSNO and CWSNO, and extracts the PCA features with three ways such as mean, median and mode which are then classified with MD classifier using LOOCV of R-SM to recognize the faces. The performance metrics is also computed and compared. Performance metrics of the proposed approach and the current approach are computed and compared. Thus, the suggested method is useful for increasing the visibility of facial recognition, and overcoming a pose, similarity and illumination problem, which provides a more accurate investigation of the required recognition procedures.

Managing the Economic Challenges in the Treatment of Heart Failure

The economics of heart failure (HF) touches all patients with HF, their families, and the physicians and health systems that care for them. HF is specifically targeted by cost-reduction and care quality initiatives from the Centers for Medicare and Medicaid Services (CMS). The changing quality assessment and payment landscape is, and will continue to be, challenging for hospitals and HF specialists as they provide care for patients with this debilitating disease. Quality-based payment systems with evolving performance metrics are replacing traditional volume-based fee-for-service models. A critical objective of quality-based models is to improve care and reduce cost, but there are few data to support decision-making on how to improve. CMS payment programs and their implications for health systems treating HF were reviewed at a symposium at the Heart Failure Society of America conference in Nashville, Tennessee on September 15, 2018. This article constitutes the proceedings from that symposium.

Development of a standardised set of metrics for monitoring site performance in multicentre randomised trials: a Delphi study

BACKGROUND

Site performance is key to the success of large multicentre randomised trials. A standardised set of clear and accessible summaries of site performance could facilitate the timely identification and resolution of potential problems, minimising their impact. The aim of this study was to identify and agree a core set of key performance metrics for managing multicentre randomised trials.

METHODS

We used a mixed methods approach to identify potential metrics and to achieve consensus about the final set, adapting methods that are recommended by the COMET Initiative for developing core outcome sets in health care. We used performance metrics identified from our systematic search and focus groups to create an online Delphi survey. We invited respondents to score each metric for inclusion in the final core set, over three survey rounds. Metrics scored as critical by ≥70% and unimportant by <15% of respondents were taken forward to a consensus meeting of representatives from key UK-based stakeholders. Participants in the consensus meeting discussed and voted on each metric, using anonymous electronic voting. Metrics with >50% of participants voting for inclusion were retained.

RESULTS

Round 1 of the Delphi survey presented 28 performance metrics, and a further six were added in round 2. Of 294 UK-based stakeholders who registered for the Delphi survey, 211 completed all three rounds. At the consensus meeting, 17 metrics were discussed and voted on: 15 metrics were retained following survey round 3, plus two others that were preferred by consensus meeting participants. Consensus was reached on a final core set of eight performance metrics in three domains: (1) recruitment and retention, (2) data quality and (3) protocol compliance. A simple tool for visual reporting of the metrics is available from the Nottingham Clinical Trials Unit website.

CONCLUSIONS

We have established a core set of metrics for measuring the performance of sites in multicentre randomised trials. These metrics could improve trial conduct by enabling researchers to identify and address problems before trials are adversely affected. Future work could evaluate the effectiveness of using the metrics and reporting tool.

Monitoring performance of sites within multicentre randomised trials: a systematic review of performance metrics

Background

Large multicentre trials are complex and expensive projects. A key factor for their successful planning and delivery is how well sites meet their targets in recruiting and retaining participants, and in collecting high-quality, complete data in a timely manner. Collecting and monitoring easily accessible data relevant to performance of sites has the potential to improve trial management efficiency. The aim of this systematic review was to identify metrics that have either been proposed or used for monitoring site performance in multicentre trials.

Methods

We searched the Cochrane Library, five biomedical bibliographic databases (CINAHL, EMBASE, Medline, PsychINFO and SCOPUS) and Google Scholar for studies describing ways of monitoring or measuring individual site performance in multicentre randomised trials. Records identified were screened for eligibility. For included studies, data on study content were extracted independently by two reviewers, and disagreements resolved by discussion.

Results

After removing duplicate citations, we identified 3188 records. Of these, 21 were eligible for inclusion and yielded 117 performance metrics. The median number of metrics reported per paper was 8, range 1–16. Metrics broadly fell into six categories: site potential; recruitment; retention; data collection; trial conduct and trial safety.

Conclusions

This review identifies a list of metrics to monitor site performance within multicentre randomised trials. Those that would be easy to collect, and for which monitoring might trigger actions to mitigate problems at site level, merit further evaluation.

A case-study based framework for assessing the multi-sector performance of green infrastructure

Green infrastructure is emerging as a holistic stormwater management strategy that can also provide multi-sector benefits. Robust demonstration of project success can help leverage the appeal of green infrastructure to different sectors and open the door to a variety of funding opportunities. Yet comprehensively assessing the performance of these natural systems can be challenging, especially when communicating the benefits to a wide variety of stakeholders. A cohesive, well-described assessment structure may promote a higher degree of investor confidence by more comprehensively monitoring and measuring green infrastructure success. This paper develops a conceptual framework that incorporates a robust assessment component for communicating with potential investors through the inclusion of multiple evaluation methods, performance metrics, and risk categories. The applied performance of this framework is then validated using fourteen U.S. and international case studies. We found that our framework fit a wide range of projects while maintaining a degree of flexibility that did not sacrifice specificity when applied to individual case studies. This suggests that: 1) some GI projects already incorporate one or more evaluation methods; 2) a number of highly specific metrics-particularly social and economic performance metrics-exist that are capable of capturing a wide-range of benefits that can be easily integrated into a framework; 3) the incorporation of risk and risk management technique identification could be emphasized to increase investor confidence; 4) at least some degree of standardization across projects exists already which can help future project implementers design GI strategies that best fit their needs.

American College of Radiology Accreditation, Performance Metrics, Reimbursement, and Economic Considerations in Breast MR Imaging

Accreditation through the American College of Radiology (ACR) Breast Magnetic Resonance Imaging Accreditation Program is necessary to qualify for reimbursement from Medicare and many private insurers and provides facilities with peer review on image acquisition and clinical quality. Adherence to ACR quality control and technical practice parameter guidelines for breast MR imaging and performance of a medical outcomes audit program will maintain high-quality imaging and facilitate accreditation. Economic factors likely to influence the practice of breast MR imaging include cost-effectiveness, competition with lower-cost breast-imaging modalities, and price transparency, all of which may lower the cost of MR imaging and allow for greater utilization.

Data on performance prediction for cloud service selection

This paper contains data on Performance Prediction for Cloud Service Selection. To measure the performance metrics of any system you need to analyze the features that affect these performance, these features are called " workload parameters". The data described here is collected from the KSA Ministry of Finance that contains 28,147 instances from 13 cloud nodes. It was recorded during the period from March 1, 2016, to February 20, 2017, in continuous time slots. In this article we selected 9 workload parameters: Number of Jobs in a Minute, Number of Jobs in 5 min, Number of Jobs in 15 min, Memory Capacity, Disk Capacity,: Number of CPU Cores, CPU Speed per Core, Average Receive for Network Bandwidth in Kbps and Average Transmit for Network Bandwidth in Kbps. Moreover, we selected 3 performance metrics: Memory utilization, CPU utilization and response time in milliseconds. This data article is related to the research article titled "An Automated Performance Prediction Model for Cloud Service Selection from Smart Data" (Al-Faifi et al., 2018) [1].

Trends in engagement in the cascade of care for opioid use disorder, Vancouver, Canada, 2006-2016

BACKGROUND

A cascade of care framework has been proposed to identify and address implementation gaps in addiction medicine. Using this framework, we characterized temporal trends in engagement in care for opioid use disorder (OUD) in Vancouver, Canada.

METHODS

Using data from two cohorts of people who use drugs, we assessed the yearly proportion of daily opioid users achieving four sequential stages of the OUD cascade of care [linkage to addiction care; linkage to opioid agonist treatment (OAT); retention in OAT; and stability] between 2006 and 2016. We evaluated temporal trends of cascade indicators, adjusting for socio-demographic characteristics, HIV/HCV status, substance use patterns, and social-structural exposures.

RESULTS

We included 1615 daily opioid users. Between 2006 and 2016, we observed improvements in linkage to care (from 73.2% to 78.9%, p = <0.001), linkage to (from 69.2% to 70.6%, p = 0.011) and retention in OAT (from 29.1% to 35.5%, p = <0.001), and stability (from 10.4% to 17.1%, p = <0.001). In adjusted analyses, later calendar year of observation was associated with increased odds of linkage to care (Adjusted Odds Ratio [AOR] = 1.02, 95% Confidence Interval [CI]: 1.01-1.04), retention in OAT (AOR 1.02, 95% CI: 1.01-1.04) and stability (AOR = 1.03, 95% CI: 1.01-1.05), but not with linkage to OAT (AOR 1.00, 95% CI: 0.98-1.01).

CONCLUSIONS

Temporal improvements in OUD cascade of care indicators were observed. However, only a third of participants were retained in OAT in 2016. These findings suggest the need for novel approaches to improve engagement in care for OUD to address the escalating opioid-related overdose crisis.

Performance Improvement in Head and Neck Cancer

Performance improvement requires establishing a platform to set benchmarks and monitor the quality of care provided through quality indicators and metrics. This has long been recognized as critical to overall quality improvement and more recently, has become federally mandated. Here, we review recent studies evaluating performance in head and neck cancer care, from those spanning all phases of head and neck cancer care to others focused on head and neck surgical performance, including both national and departmental/institutional efforts.

Quantification and visualization of coordination during non-cyclic upper extremity motion

There are many design challenges in creating at-home tele-monitoring systems that enable quantification and visualization of complex biomechanical behavior. One such challenge is robustly quantifying joint coordination in a way that is intuitive and supports clinical decision-making. This work defines a new measure of coordination called the relative coordination metric (RCM) and its accompanying normalization schemes. RCM enables quantification of coordination during non-constrained discrete motions. Here RCM is applied to a grasping task. Fifteen healthy participants performed a reach, grasp, transport, and release task with a cup and a pen. The measured joint angles were then time-normalized and the RCM time-series were calculated between the shoulder-elbow, shoulder-wrist, and elbow-wrist. RCM was normalized using four differing criteria: the selected joint degree of freedom, angular velocity, angular magnitude, and range of motion. Percent time spent in specified RCM ranges was used asa composite metric and was evaluated for each trial. RCM was found to vary based on: (1) chosen normalization scheme, (2) the stage within the task, (3) the object grasped, and (4) the trajectory of the motion. The RCM addresses some of the limitations of current measures of coordination because it is applicable to discrete motions, does not rely on cyclic repetition, and uses velocity-based measures. Future work will explore clinically relevant differences in the RCM as it is expanded to evaluate different tasks and patient populations.

A novel ECG detector performance metric and its relationship with missing and false heart rate limit alarms

PURPOSE

Performance of ECG beat detectors is traditionally assessed on long intervals (e.g.: 30min), but only incorrect detections within a short interval (e.g.: 10s) may cause incorrect (i.e., missed+false) heart rate limit alarms (tachycardia and bradycardia). We propose a novel performance metric based on distribution of incorrect beat detection over a short interval and assess its relationship with incorrect heart rate limit alarm rates.

BASIC PROCEDURES

Six ECG beat detectors were assessed using performance metrics over long interval (sensitivity and positive predictive value over 30min) and short interval (Area Under empirical cumulative distribution function (AUecdf) for short interval (i.e., 10s) sensitivity and positive predictive value) on two ECG databases. False heart rate limit and asystole alarm rates calculated using a third ECG database were then correlated (Spearman's rank correlation) with each calculated performance metric.

MAIN FINDINGS

False alarm rates correlated with sensitivity calculated on long interval (i.e., 30min) (ρ=-0.8 and p<0.05) and AUecdf for sensitivity (ρ=0.9 and p<0.05) in all assessed ECG databases. Sensitivity over 30min grouped the two detectors with lowest false alarm rates while AUecdf for sensitivity provided further information to identify the two beat detectors with highest false alarm rates as well, which was inseparable with sensitivity over 30min.

PRINCIPAL CONCLUSIONS

Short interval performance metrics can provide insights on the potential of a beat detector to generate incorrect heart rate limit alarms.

ESR concept paper on value-based radiology

The European Society of Radiology (ESR) established a Working Group on Value-Based Imaging (VBI WG) in August 2016 in response to developments in European healthcare systems in general, and the trend within radiology to move from volume- to value-based practice in particular. The value-based healthcare (VBH) concept defines "value" as health outcomes achieved for patients relative to the costs of achieving them. Within this framework, value measurements start at the beginning of therapy; the whole diagnostic process is disregarded, and is considered only if it is the cause of errors or complications. Making the case for a new, multidisciplinary organisation of healthcare delivery centred on the patient, this paper establishes the diagnosis of disease as a first outcome in the interrelated activities of the healthcare chain. Metrics are proposed for measuring the quality of radiologists' diagnoses and the various ways in which radiologists provide value to patients, other medical specialists and healthcare systems at large. The ESR strongly believes value-based radiology (VBR) is a necessary complement to existing VBH concepts. The Society is determined to establish a holistic VBR programme to help European radiologists deal with changes in the evolution from volume- to value-based evaluation of radiological activities. Main Messages • Value-based healthcare defines value as patient's outcome over costs. • The VBH framework disregards the diagnosis as an outcome. • VBH considers diagnosis only if wrong or a cause of complications. • A correct diagnosis is the first outcome that matters to patients. • Metrics to measure radiologists' impacts on patient outcomes are key. • The value provided by radiology is multifaceted, going beyond exam volumes.

Analysis of Subjects' Vulnerability in a Touch Screen Game Using Behavioral Metrics

In this article, we report results on an experimental study conducted with volunteer subjects playing a touch-screen game with two unique difficulty levels. Subjects have knowledge about the rules of both game levels, but only sufficient playing experience with the easy level of the game, making them vulnerable with the difficult level. Several behavioral metrics associated with subjects' playing the game are studied in order to assess subjects' mental-workload changes induced by their vulnerability. Specifically, these metrics are calculated based on subjects' finger kinematics and decision making times, which are then compared with baseline metrics, namely, performance metrics pertaining to how well the game is played and a physiological metric called pnn50 extracted from heart rate measurements. In balanced experiments and supported by comparisons with baseline metrics, it is found that some of the studied behavioral metrics have the potential to be used to infer subjects' mental workload changes through different levels of the game. These metrics, which are decoupled from task specifics, relate to subjects' ability to develop strategies to play the game, and hence have the advantage of offering insight into subjects' task-load and vulnerability assessment across various experimental settings.

Measuring transplant center performance: The goals are not controversial but the methods and consequences can be

PURPOSE OF REVIEW

Risks of regulatory scrutiny has generated widespread concern about increasingly risk averse transplant center behaviors regarding both donor and candidate acceptance patterns. To address potential unintended consequences threatening access to care, we discuss recent changes in regulatory metrics and potential improvements in quality oversight of transplant centers.

RECENT FINDINGS

Despite many recent changes to one-year patient and graft survival regulatory criteria, the capacity to accurately identify true underperforming centers and avoiding false positive flagging remains an area of great concern. Numerous studies have demonstrated restrictions in transplant volume and access following transplant center flagging.

SUMMARY

Current regulatory criteria are limited in their capacity to accurately identify poorly performing centers and potentially encourage risk-averse behavior by transplant centers. Efforts to address these concerns should focus on (1) improving risk-adjustment models with better data which captures the acuity of candidate and donor risk, (2) reconsidering primary outcomes measured to assess comprehensive transplant center performance, (3) improving education to address rational or perceived disincentives, and (4) using data more effectively to share best practices.

Proctors exploit three-dimensional ghost tools during clinical-like training scenarios: a preliminary study

Purpose

In this study, we examine three-dimensional (3D) proctoring tools (i.e., semitransparent ghost tools overlaid on the surgeon’s field of view) on realistic surgical tasks. Additionally, we develop novel, quantitative measures of whether proctors exploit the additional capabilities offered by ghost tools.

Methods

Seven proctor–trainee pairs completed realistic surgical tasks such as tissue dissection and suturing in a live porcine model using 3D ghost tools on the da Vinci Xi Surgical System. The usability and effectiveness of 3D ghost tools were evaluated using objective measures of proctor performance based on proctor hand movements and button presses, as well as post-study questionnaires.

Results

Proctors exploited the capabilities of ghost tools, such as 3D hand movement (p < 0.001), wristedness (p < 0.001), finger pinch gestures (p < 0.001), and bimanual hand motions (p < 0.001). The median ghost tool excursion distances across proctors in the x-, y-, and z-directions were 57.6, 31.9, and 50.7, respectively. Proctors and trainees consistently evaluated the ghost tools as effective across multiple categories of mentoring. Trainees found ghost tools more helpful than proctors across all categories (p < 0.05).

Conclusions

Proctors exploit the augmented capabilities of 3D ghost tools during clinical-like training scenarios. Additionally, both proctors and trainees evaluated ghost tools as effective mentoring tools, thereby confirming previous studies on simple, inanimate tasks. Based on this preliminary work, advanced mentoring technologies, such as 3D ghost tools, stand to improve current telementoring and training technologies in robot-assisted minimally invasive surgery.

Electronic supplementary material

The online version of this article (doi:10.1007/s00345-016-1944-x) contains supplementary material, which is available to authorized users.

Viewpoint matters: objective performance metrics for surgeon endoscope control during robot-assisted surgery

Background

Effective visualization of the operative field is vital to surgical safety and education. However, additional metrics for visualization are needed to complement other common measures of surgeon proficiency, such as time or errors. Unlike other surgical modalities, robot-assisted minimally invasive surgery (RAMIS) enables data-driven feedback to trainees through measurement of camera adjustments. The purpose of this study was to validate and quantify the importance of novel camera metrics during RAMIS.

Methods

New (n = 18), intermediate (n = 8), and experienced (n = 13) surgeons completed 25 virtual reality simulation exercises on the da Vinci Surgical System. Three camera metrics were computed for all exercises and compared to conventional efficiency measures.

Results

Both camera metrics and efficiency metrics showed construct validity (p < 0.05) across most exercises (camera movement frequency 23/25, camera movement duration 22/25, camera movement interval 19/25, overall score 24/25, completion time 25/25). Camera metrics differentiated new and experienced surgeons across all tasks as well as efficiency metrics. Finally, camera metrics significantly (p < 0.05) correlated with completion time (camera movement frequency 21/25, camera movement duration 21/25, camera movement interval 20/25) and overall score (camera movement frequency 20/25, camera movement duration 19/25, camera movement interval 20/25) for most exercises.

Conclusions

We demonstrate construct validity of novel camera metrics and correlation between camera metrics and efficiency metrics across many simulation exercises. We believe camera metrics could be used to improve RAMIS proficiency-based curricula.

Moments and Root-Mean-Square Error of the Bayesian MMSE Estimator of Classification Error in the Gaussian Model

The most important aspect of any classifier is its error rate, because this quantifies its predictive capacity. Thus, the accuracy of error estimation is critical. Error estimation is problematic in small-sample classifier design because the error must be estimated using the same data from which the classifier has been designed. Use of prior knowledge, in the form of a prior distribution on an uncertainty class of feature-label distributions to which the true, but unknown, feature-distribution belongs, can facilitate accurate error estimation (in the mean-square sense) in circumstances where accurate completely model-free error estimation is impossible. This paper provides analytic asymptotically exact finite-sample approximations for various performance metrics of the resulting Bayesian Minimum Mean-Square-Error (MMSE) error estimator in the case of linear discriminant analysis (LDA) in the multivariate Gaussian model. These performance metrics include the first, second, and cross moments of the Bayesian MMSE error estimator with the true error of LDA, and therefore, the Root-Mean-Square (RMS) error of the estimator. We lay down the theoretical groundwork for Kolmogorov double-asymptotics in a Bayesian setting, which enables us to derive asymptotic expressions of the desired performance metrics. From these we produce analytic finite-sample approximations and demonstrate their accuracy via numerical examples. Various examples illustrate the behavior of these approximations and their use in determining the necessary sample size to achieve a desired RMS. The Supplementary Material contains derivations for some equations and added figures.