A Machine Learning Approach with Human-AI Collaboration for Automated Classification of Patient Safety Event Reports: Algorithm Development and Validation Study

BACKGROUND

Adverse events refer to incidents with potential or actual harm to patients in hospitals. These events are typically documented through patient safety event (PSE) reports, which consist of detailed narratives providing contextual information on the occurrences. Accurate classification of PSE reports is crucial for patient safety monitoring. However, this process faces challenges due to inconsistencies in classifications and the sheer volume of reports. Recent advancements in text representation, particularly contextual text representation derived from transformer-based language models, offer a promising solution for more precise PSE report classification. Integrating the machine learning (ML) classifier necessitates a balance between human expertise and artificial intelligence (AI). Central to this integration is the concept of explainability, which is crucial for building trust and ensuring effective human-AI collaboration.

OBJECTIVE

This study aims to investigate the efficacy of ML classifiers trained using contextual text representation in automatically classifying PSE reports. Furthermore, the study presents an interface that integrates the ML classifier with the explainability technique to facilitate human-AI collaboration for PSE report classification.

METHODS

This study used a data set of 861 PSE reports from a large academic hospital's maternity units in the Southeastern United States. Various ML classifiers were trained with both static and contextual text representations of PSE reports. The trained ML classifiers were evaluated with multiclass classification metrics and the confusion matrix. The local interpretable model-agnostic explanations (LIME) technique was used to provide the rationale for the ML classifier's predictions. An interface that integrates the ML classifier with the LIME technique was designed for incident reporting systems.

RESULTS

The top-performing classifier using contextual representation was able to obtain an accuracy of 75.4% (95/126) compared to an accuracy of 66.7% (84/126) by the top-performing classifier trained using static text representation. A PSE reporting interface has been designed to facilitate human-AI collaboration in PSE report classification. In this design, the ML classifier recommends the top 2 most probable event types, along with the explanations for the prediction, enabling PSE reporters and patient safety analysts to choose the most suitable one. The LIME technique showed that the classifier occasionally relies on arbitrary words for classification, emphasizing the necessity of human oversight.

CONCLUSIONS

This study demonstrates that training ML classifiers with contextual text representations can significantly enhance the accuracy of PSE report classification. The interface designed in this study lays the foundation for human-AI collaboration in the classification of PSE reports. The insights gained from this research enhance the decision-making process in PSE report classification, enabling hospitals to more efficiently identify potential risks and hazards and enabling patient safety analysts to take timely actions to prevent patient harm.

[The World Health Organization: Project of novels in the field of controlling international medical sanitary regulations]

The article considers draft of new edition of the International Health Regulations. The possible risks associated with changing of the document are evaluated from perspective of member countries, in whose territories occur or supposedly occur emergency situations in field of public health having international importance.

Protection of Animals during Transport: Analysis of the Infringements Reported from 2009 to 2013 during On-Road Inspections in Italy

Simple Summary

On-road inspections of vehicles that transport animals are mandatory in Europe. Infringements of the Council Regulation (EC) 1/2005 that were ascertained by competent authorities during on-road inspections published by the Italian Health Ministry from 2009 to 2013 were analyzed. The aims were both to identify possible routes or species that are more likely to be at risk of poor welfare conditions and to suggest recommendations. A total of 985 infringements were reported. For analysis, they were split into three main categories that were related to animal welfare (AW), vehicle (V), and accompanying documents (D). Each category was further classified under different subcategories (e.g., overcrowding for AW, lack of drinking system for V, and lack of health certificate for D). The most frequent infringements were related to D (34.4%), but more than one infringement was often found during an inspection (mean: 1.58; max: 9). A score (from 1 to 3) that was related to the severity of the animal-welfare issues was created which was found to be associated with year, species, authority, and country of dispatch (p < 0.001). Over the years, the only improvement was in the accompanying documentation. Vehicles that were transporting pigs, sheep, or goats were more likely to have the poorest welfare conditions, whilst vehicles that were transporting horses or other species, including dogs, were often found with irregular documentation. AW infringements were more likely to be uncovered during road inspections where traffic police and veterinarians worked together. This type of road inspection should be intensified so as to enhance animal welfare during transportation.

Abstract

Council Regulation (EC) No 1/2005 requires that vehicles that are transporting animals be subjected to checks conducted by competent authorities. Yearly, each member state sends a report to the European government on the infringements that have been discovered during on-road inspections. The reports that were published by the Italian Ministry of Public Health from 2009 to 2013 were analyzed. Possible associations between the type of infringement (related to animal welfare (AW), vehicle (V) and accompanying documents (D)), year, season, transported species, place of inspection, and competent authorities were identified. A total of 985 infringements were analyzed, with some vehicles receiving more than one (mean: 1.58; max: 9). A score (from 1 to 3) that was related to the severity of the infringements was created. In 2009 and 2010, there was a 50% higher probability of encountering penalties of a lower severity (D or V) than in 2011 (p < 0.0001). Vehicles that were transporting pigs showed the highest probability of committing animal welfare-related infringements (odds ratio (OR) = 3.85, 95% confidence interval (95%CI) = 1.82–8.76, p < 0.0001). Vehicles were four times more likely to suffer animal welfare-related penalties when traffic police worked in synergy with veterinary services (OR = 4.12, 95%CI = 1.70–11.13, p = 0.0005). Vehicles that were transporting Equidae and “other species,” including pets, for commercial purposes were more likely to be fined for a lack or incompleteness of the veterinary documents than those transporting cattle (p = 0.002 and p = 0.004, respectively). This study gives statistical evidence of the implementation of EC 1/2005. The training of transporters and drivers on how to manage transport in an animal welfare-friendly manner and a standardized method on how to conduct road inspections among competent authorities are recommended.

[Eleven thesis on the archive of scientific research, for a new patrimonial and scientific policy]

Abstracting the main content of a recent report on the bad state of the archives of scientific research, this paper puts forward eleven thesis likely to feed, in this time of numeric transition to a new documentary regime and to a new patrimonial policy. The recent numeric conditions impose to set new archival pratices, more proactive, anticipative and prospective. Archives of scientific research must be thought in a double memorial and scientific dimension, and not only as a patrimonial or historical one.