Ensuring the Robustness and Reliability of Data-Driven Knowledge Discovery Models in Production and Manufacturing

Machine Learning Model Drift: Predicting Diagnostic Imaging Follow-Up as a Case Example

OBJECTIVE

Address model drift in a machine learning (ML) model for predicting diagnostic imaging follow-up using data augmentation with more recent data versus retraining new predictive models.

METHODS

This institutional review board-approved retrospective study was conducted January 1, 2016, to December 31, 2020, at a large academic institution. A previously trained ML model was trained on 1,000 radiology reports from 2016 (old data). An additional 1,385 randomly selected reports from 2019 to 2020 (new data) were annotated for follow-up recommendations and randomly divided into two sets: training (n = 900) and testing (n = 485). Support vector machine and random forest (RF) algorithms were constructed and trained using 900 new data reports plus old data (augmented data, new models) and using only new data (new data, new models). The 2016 baseline model was used as comparator as is and trained with augmented data. Recall was compared with baseline using McNemar's test.

RESULTS

Follow-up recommendations were contained in 11.3% of reports (157 or 1,385). The baseline model retrained with new data had precision = 0.83 and recall = 0.54; none significantly different from baseline. A new RF model trained with augmented data had significantly better recall versus the baseline model (0.80 versus 0.66, P = .04) and comparable precision (0.90 versus 0.86).

DISCUSSION

ML methods for monitoring follow-up recommendations in radiology reports suffer model drift over time. A newly developed RF model achieved better recall with comparable precision versus simply retraining a previously trained original model with augmented data. Thus, regularly assessing and updating these models is necessary using more recent historical data.

Prognostic modeling of predictive maintenance with survival analysis for mobile work equipment

In recent years there is a data surge of industrial and business data. This posses opportunities and challenges at the same time because the wealth of information is usually buried in complex and frequently disconnected data sets. Predictive maintenance utilizes such data for developing prognostic and diagnostic models that allow the optimization of the life cycle of machine components. In this paper, we address the modeling of the prognostics of machine components from mobile work equipment. Specifically, we are estimating survival curves and hazard rates using parametric and non-parametric models to characterize time dependent failure probabilities of machine components. As a result, we find the presence of different types of censoring masking the presence of different populations that can cause severe problems for statistical estimators and the interpretations of results. Furthermore, we show that the obtained hazard functions for different machine components are complex and versatile and are best modeled via non-parametric estimators. However, notable exceptions for individual machine components can be found amenable for a Generalized-gamma and Weibull model.

The Contribution of Data-Driven Technologies in Achieving the Sustainable Development Goals

The United Nations’ Sustainable Development Goals (SDGs) set out to improve the quality of life of people in developed, emerging, and developing countries by covering social and economic aspects, with a focus on environmental sustainability. At the same time, data-driven technologies influence our lives in all areas and have caused fundamental economical and societal changes. This study presents a comprehensive literature review on how data-driven approaches have enabled or inhibited the successful achievement of the 17 SDGs to date. Our findings show that data-driven analytics and tools contribute to achieving the 17 SDGs, e.g., by making information more reliable, supporting better-informed decision-making, implementing data-based policies, prioritizing actions, and optimizing the allocation of resources. Based on a qualitative content analysis, results were aggregated into a conceptual framework, including the following categories: (1) uses of data-driven methods (e.g., monitoring, measurement, mapping or modeling, forecasting, risk assessment, and planning purposes), (2) resulting positive effects, (3) arising challenges, and (4) recommendations for action to overcome these challenges. Despite positive effects and versatile applications, problems such as data gaps, data biases, high energy consumption of computational resources, ethical concerns, privacy, ownership, and security issues stand in the way of achieving the 17 SDGs.

The temporal inference with the use of ant-based clustering algorithm and flow graphs in the problem of prognosing complications of medical surgical procedures

In the era of a rapidly aging European society, the demand for proven clinical decision support systems, links health observations with medical knowledge in order to assist clinicians in decision making is constantly growing. An increasing problem for this type of systems is not only the size of the processed data sets but also the heterogeneity of these data. Clinical forecasting often requires processing of both numerical data and multi-category data which are temporal. The conducted research has shown that a good solution to this problem may lie in the use of temporal inference, the ant-based clustering algorithm, rough sets, and fuzzy sets. The experiments used a real set of medical data representing cases of a disease that significantly reduces a woman's quality of life. Each case of uterine myoma disease (which affects more than 50% of women over the age of 35) is represented by more than 140 heterogeneous features. An incorrect decision about the type of surgery (thermoablation or surgery) not only affects female fertility but also the high risk of complications. Therefore, the solution discussed in this paper may turn out to be extremely important.

Infoecology of the deep learning and smart manufacturing: thematic and concept interactions

Purpose

This infecological study mainly aimed to know the thematic and conceptual relationship in published papers in deep learning (DL) and smart manufacturing (SM).

Design/methodology/approach

The research methodology has specific research objectives based on the type and method of research, data analysis tools. In general, description methods are applied by Web of Science (WoS) analysis tools and Voyant tools as a web-based reading and analysis environment for digital texts. The Yewno tool is applied to draw a knowledge map to show the concept's interaction between DL and SM.

Findings

The knowledge map of DL and SM concepts shows that there are currently few concepts interacting with each other, while the rapid growth of technology and the needs of today's society have revealed the need to use more and more DL in SM. The results of this study can provide a coherent and well-mapped road map to the main policymakers of the field of research in DL and SM, through the study of coexistence and interactions of the thematic categories with other thematic areas. In this way, they can design more effective guidelines and strategies to solve the problems of researchers in conducting their studies and direct. The analysis results demonstrated that the information ecosystem of DL and SM studies dynamically developed over time. The continuous conduction flow of scientific studies in this field brought continuous changes into the infoecology of subjects and concepts in this area.

Originality/value

The paper investigated the thematic interaction of the scientific productions in DL and SM and discussed possible implications. We used of the variety tools and techniques to draw our own perspective. Also, we drew arguments from other research work to back up our findings.