Digital Health Education for the Future: The SaNuRN (Santé Numérique Rouen-Nice) Consortium's Journey

SaNuRN is a five-year project by the University of Rouen Normandy (URN) and the Côte d’Azur University (CAU) consortium to optimize digital health education for medical and paramedical students, professionals, and administrators. The project includes a skills framework, training modules, and teaching resources. In 2027, SaNuRN is expected to train a significant portion of the 400,000 health and paramedical professions students at the French national level. Our purpose is to give a synopsis of the SaNuRN initiative, emphasizing its novel educational methods and how they will enhance the delivery of digital health education. Our goals include showcasing SaNuRN as a comprehensive program consisting of a proficiency framework, instructional modules, and educational materials and explaining how SaNuRN is implemented in the participating academic institutions. SaNuRN is a project aimed at educating and training health-related and paramedics students in digital health. The project results from a cooperative effort between URN and CAU, covering four French departments. The project is based on the French National Referential on Digital Health (FNRDH), which defines the skills and competencies to be acquired and validated by every student in the health, paramedical, and social professions curricula. The SaNuRN team is currently adapting the existing URN and CAU syllabi to FNRDH and developing short-duration video capsules of 20 to 30 minutes to teach all the relevant material. The project aims to ensure that the largest student population earns the necessary skills, and it has developed a two-tier system involving facilitators who will enable the efficient expansion of the project’s educational outreach and support the students in learning the needed material efficiently. With a focus on real-world scenarios and innovative teaching activities integrating telemedicine devices and virtual professionals, SaNuRN is committed to enabling continuous learning for healthcare professionals in clinical practice. The SaNuRN team introduced new ways of evaluating healthcare professionals by shifting from a knowledge-based to a competencies-based evaluation, aligning with the Miller teaching pyramid and using the Objective Structured Clinical Examination and Script Concordance Test in digital health education. Drawing on the expertise of URN, CAU, and their public health and digital research laboratories and partners, the SaNuRN project represents a platform for continuous innovation, including telemedicine training and living labs with virtual and interactive professional activities. The SaNuRN project provides a comprehensive, personalized 30-hour training package for health and paramedical students, addressing all 70 FNRDH competencies. The program is enhanced using AI and NLP to create virtual patients and professionals for digital healthcare simulation. SaNuRN teaching materials are open-access. The project collaborates with academic institutions worldwide to develop educational material in digital health in English and multilingual formats. SaNuRN offers a practical and persuasive training approach to meet the current digital health education requirements.

Potential efficacy of T and B lymphocyte-targeted therapies on articular involvement of patients with rheumatoid arthritis and systemic sclerosis overlap syndrome. Results from a 2-centre series of 22 cases

OBJECTIVES

To analyse in routine practice the efficacy of targeted therapies on joint involvement of patients with rheumatoid arthritis/systemic sclerosis (RA/SSc) overlap syndrome.

METHODS

This was a retrospective analysis of medical records of two academic centres over a 10-year period. Joint response to targeted therapies was measured according to EULAR criteria based on Disease Activity Score (DAS)-28. In addition, changes in CRP level and glucocorticoid consumption were recorded.

RESULTS

Nineteen patients were included. Methotrexate (n=11) and hydroxychloroquine (n=4) were the most used first-line treatments. Targeted therapies were frequently used (n=14). Tocilizumab was the most selected therapy (n=8), then rituximab (n=5), abatacept and anti-tumour necrosis factor (n=4). Twenty-one treatment sequences were assessed, including 18 with EULAR response criteria. Responses were "good" or "moderate" in 100% (4/4) of patients treated with abatacept, 80% (4/5) with rituximab, 40% (2/5) with tocilizumab, and 25% (1/4) with anti-TNF. T and B lymphocyte-targeted therapies (abatacept, rituximab) resulted more frequently in a "good" or "moderate" response compared to cytokine inhibitors (tocilizumab, etanercept, infliximab) with a significant decrease in DAS-28 at 6 months (-1.75; p=0.016) and a trend to a lower consumption of glucocorticoids.

CCONCLUSIONS

In patients with RA/SSc overlap syndrome refractory to conventional synthetic-DMARDs, T and B lymphocyte-targeted therapies seem to be a promising therapeutic option to control joint activity.

Sharing Digital Health Educational Resources in a One-Stop Shop Portal: Tutorial on the Catalog and Index of Digital Health Teaching Resources (CIDHR) Semantic Search Engine

BACKGROUND

Access to reliable and accurate digital health web-based resources is crucial. However, the lack of dedicated search engines for non-English languages, such as French, is a significant obstacle in this field. Thus, we developed and implemented a multilingual, multiterminology semantic search engine called Catalog and Index of Digital Health Teaching Resources (CIDHR). CIDHR is freely accessible to everyone, with a focus on French-speaking resources. CIDHR has been initiated to provide validated, high-quality content tailored to the specific needs of each user profile, be it students or professionals.

OBJECTIVE

This study's primary aim in developing and implementing the CIDHR is to improve knowledge sharing and spreading in digital health and health informatics and expand the health-related educational community, primarily French speaking but also in other languages. We intend to support the continuous development of initial (ie, bachelor level), advanced (ie, master and doctoral levels), and continuing training (ie, professionals and postgraduate levels) in digital health for health and social work fields. The main objective is to describe the development and implementation of CIDHR. The hypothesis guiding this research is that controlled vocabularies dedicated to medical informatics and digital health, such as the Medical Informatics Multilingual Ontology (MIMO) and the concepts structuring the French National Referential on Digital Health (FNRDH), to index digital health teaching and learning resources, are effectively increasing the availability and accessibility of these resources to medical students and other health care professionals.

METHODS

First, resource identification is processed by medical librarians from websites and scientific sources preselected and validated by domain experts and surveyed every week. Then, based on MIMO and FNRDH, the educational resources are indexed for each related knowledge domain. The same resources are also tagged with relevant academic and professional experience levels. Afterward, the indexed resources are shared with the digital health teaching and learning community. The last step consists of assessing CIDHR by obtaining informal feedback from users.

RESULTS

Resource identification and evaluation processes were executed by a dedicated team of medical librarians, aiming to collect and curate an extensive collection of digital health teaching and learning resources. The resources that successfully passed the evaluation process were promptly included in CIDHR. These resources were diligently indexed (with MIMO and FNRDH) and tagged for the study field and degree level. By October 2023, a total of 371 indexed resources were available on a dedicated portal.

CONCLUSIONS

CIDHR is a multilingual digital health education semantic search engine and platform that aims to increase the accessibility of educational resources to the broader health care-related community. It focuses on making resources "findable," "accessible," "interoperable," and "reusable" by using a one-stop shop portal approach. CIDHR has and will have an essential role in increasing digital health literacy.

Omalizumab Drug Survival in Chronic Urticaria: A Retrospective Multicentric French Study

BACKGROUND

Omalizumab (OMA) dramatically improves disease control and quality of life in patients with chronic urticaria (CU).

OBJECTIVE

We aimed to evaluate the discontinuation patterns of OMA and their determinants in a cohort of French patients with CU.

METHODS

We conducted a retrospective multicenter study in 9 French tertiary referral hospitals. All patients diagnosed with either spontaneous (CSU) and/or inducible (CIndU) CU who received at least 1 injection of OMA between 2009 and 2021 were included. We analyzed OMA drug survival and investigated possible determinants using Kaplan-Meier curves and log-rank tests.

RESULTS

A total of 878 patients were included in this study; 48.8% had CSU, 10.1% CIndU, and 41.1% a combination of both. OMA was discontinued in 408 patients, but the drug was later reintroduced in 50% of them. The main reason for discontinuing treatment was the achievement of a well-controlled disease in 50% of patients. Half of the patients were still being treated with OMA 2.4 years after the initiation of treatment. Drug survival was shorter in patients with CIndU and in those with an autoimmune background. In atopic patients, OMA was discontinued earlier in patients achieving a well-controlled disease. A longer OMA drug survival was observed in patients with a longer disease duration at initiation.

CONCLUSION

In French patients with CU, the drug survival of OMA appears to be longer than that observed in previous studies conducted elsewhere, highlighting discrepancies in prescription and reimbursement possibilities. Further studies are warranted to develop customized OMA treatment schemes based on individual patterns.

Digital Emergency Management for a Complex One Health Landscape: the Need for Standardization, Integration, and Interoperability

OBJECTIVE

Planning reliable long-term planning actions to handle disruptive events requires a timely development of technological infrastructures, as well as the set-up of focused strategies for emergency management. The paper aims to highlight the needs for standardization, integration, and interoperability between Accident & Emergency Informatics (A&EI) and One Digital Health (ODH), as fields capable of dealing with peculiar dynamics for a technology-boosted management of emergencies under an overarching One Health panorama.

METHODS

An integrative analysis of the literature was conducted to draw attention to specific foci on the correlation between ODH and A&EI, in particular: (i) the management of disruptive events from private smart spaces to diseases spreading, and (ii) the concepts of (health-related) quality of life and well-being.

RESULTS

A digitally-focused management of emergency events that tackles the inextricable interconnectedness between humans, animals, and surrounding environment, demands standardization, integration, and systems interoperability. A consistent and finalized process of adoption and implementation of methods and tools from the International Standard Accident Number (ISAN), via findability, accessibility, interoperability, and reusability (FAIR) data principles, to Medical Informatics and Digital Health Multilingual Ontology (MIMO) - capable of looking at different approaches to encourage the integration between the ODH framework and the A&EI vision, provides a first answer to these needs.

CONCLUSIONS

ODH and A&EI look at different scales but with similar goals for converging health and environmental-related data management standards to enable multi-sources, interdisciplinary, and real-time data integration and interoperability. This allows holistic digital health both in routine and emergency events.

Trends in computerized provider order entry: 20-year bibliometric overview

Background

Drug-related problems (DRPs) can lead to serious health issues and have significant economic impacts on healthcare systems. One solution to address this issue is the use of computerized physician order entry systems (CPOE), which can help prevent DRPs by reducing the risk of medication errors.

Objective

The purpose of this study is to provide an analysis on scientific production of the past 20 years in order to describe trends in academic publishing on CPOE and to identify the major topics as well as the predominant actors (journals, countries) involved in this field.

Methods

A PubMed search was carried out to extract articles related to computerized provider order entry during the period January 1st 2003- December 31st 2022 using a specific query. Data were downloaded from PubMed in Extensible Markup Language (XML) and were processed through a dedicated parser.

Results

A total of 2,946 articles were retrieved among 623 journals. One third of these articles were published in eight journals. Publications grew strongly from 2002 to 2006, with a dip in 2008 followed by an increase again in 2009. After 2009, there follows a decreasing until 2022.The most producing countries are the USA with 51.39% of the publication over the period by France (3.80%), and Canada (3.77%). About disciplines, the top 3 is: "medical informatics" (21.62% of articles), "pharmacy" (19.04%), and "pediatrics" (6.56%).

Discussion

This study provides an overview of publication trends related to CPOE, which exhibited a significant increase in the first decade of the 21st century followed by a decline after 2009. Possible reasons for this decline include the emergence of digital health tools beyond CPOE, as well as healthcare professionals experiencing alert fatigue of the current system.

Conclusion

Future research should focus on analyzing publication trends in the field of medical informatics and decision-making tools to identify other areas of interest that may have surpassed the development of CPOE.

Integrating a new knowledge organisation system for monoclonal antibodies for therapeutic use authorised in Europe into HeTOP terminology-ontology server

Monoclonal antibodies (MAs) are increasingly used in the therapeutic arsenal. Clinical Data Warehouses (CDWs) offer unprecedented opportunities for research on real-word data. The objective of this work is to develop a knowledge organization system on MAs for therapeutic use (MATUs) applicable in Europe to query CDWs from a multi-terminology server (HeTOP). After expert consensus, three main health thesauri were selected: the MeSH thesaurus, the National Cancer Institute thesaurus (NCIt) and the SNOMED CT. These thesauri contain 1,723 MAs concepts, but only 99 (5.7 %) are identified as MATUs. The knowledge organisation system proposed in this article is a six-level hierarchical system according to their main therapeutic target. It includes 193 different concepts organised in a cross lingual terminology server, which will allow the inclusion of semantic extensions. Ninety nine (51.3 %) MATUs concepts and 94 (48.7 %) hierarchical concepts composed the knowledge organisation system. Two separates groups (an expert group and a validation group) carried out the selection, creation and validation processes. Queries identify, for unstructured data, 83 out of 99 (83.8 %) MATUs corresponding to 45,262 patients, 347,035 hospital stays and 427,544 health documents, and for structured data, 61 out of 99 (61.6 %) MATUs corresponding to 9,218 patients, 59,643 hospital stays and 104,737 hospital prescriptions. The volume of data in the CDW demonstrated the potential for using these data in clinical research, although not all MATUs are present in the CDW (16 missing for unstructured data and 38 for structured data). The knowledge organisation system proposed here improves the understanding of MATUs, the quality of queries and helps clinical researchers retrieve relevant medical information. The use of this model in CDW allows for the rapid identification of a large number of patients and health documents, either directly by a MATU of interest (e.g. Rituximab) but also by searching for parent concepts (e.g. Anti-CD20 Monoclonal Antibody).

Modeling and integrating interactions involving the CYP450 enzyme system in a multi-terminology server: Contribution to information extraction from a clinical data warehouse

INTRODUCTION

The cytochrome P450 (CYP450) enzyme system is involved in the metabolism of certain drugs and is responsible for most drug interactions. These interactions result in either an enzymatic inhibition or an enzymatic induction mechanism that has an impact on the therapeutic management of patients. Detecting these drug interactions will allow for better predictability in therapeutic response. Therefore, computerized solutions can represent a valuable help for clinicians in their tasks of detection.

OBJECTIVE

The objective of this study is to provide a structured data-source of interactions involving the CYP450 enzyme system. These interactions are aimed to be integrated in the cross-lingual multi-terminology server HeTOP (Health Terminologies and Ontologies Portal), to support the query processing of the clinical data warehouse (CDW) EDSaN (Entrepôt de Données de Santé Normand).

MATERIAL AND METHODS

A selection and curation of drug components (DCs) that share a relationship with the CYP450 system was performed from several international data sources. The DCs were linked according to the type of relationship which can be substrate, inhibitor, or inducer. These relationships were then integrated into the HeTOP server. To validate the CYP450 relationships, a semantic query was performed on the CDW, whose search engine is founded on HeTOP data (concepts, terms, and relations).

RESULTS

A total of 776 DCs are associated by a new interaction relationship, integrated in HeTOP, by 14 enzymes. These are CYP450 1A2, 2A6, 2B6, 2C8, 2C9, 2C18, 2C19, 2D6, 2E1, 3A4, 3A7, 11B1,11B2 mitochondrial and P-glycoprotein, constituting a total of 2,088 relationships. A general modelling of cytochromic interactions was performed. From this model, 233,006 queries were processed in less than two hours, demonstrating the usefulness and performance of our CDW implementation. Moreover, they showed that in our university hospital, the concurrent prescription that could cause a cytochromic interaction is Bisoprolol with Amiodarone by enzymatic inhibition for 2,493 patients.

DISCUSSION

The queries submitted to the CDW EDSaN allowed to highlight the most prescribed molecules simultaneously and potentially responsible for cytochromic interactions. In a second step, it would be interesting to evaluate the real clinical impact by looking for possible adverse effects of these interactions in the patients' files. Other computational solutions for cytochromic interactions exist. The impact of CYP450 is particularly important for drugs with narrow therapeutic window (NTW) as they can lead to increased toxicity or therapeutic failure. It is also important to define which drug component is a pro-drug and to considerate the many genetic polymorphisms of patients.

CONCLUSION

The HeTOP server contains a non-negligible number of relationships between drug components and CYP450 from multiple reference sources. These data allow us to query our Clinical Data Warehouse to highlight these cytochromic interactions. It would be interesting in the future to assess the actual clinical impact in hospital reports.

Appropriateness of Antibiotic Prescription During Teleconsultation

Teleconsultation has become a new means of using care which has taken off significantly since the COVID crisis, The pooling of the technological environment within the TC makes it possible to set up practice reviews by reusing the data collected. Our aim was to evaluate the relevance of antibiotic therapy during teleconsultations carried out on the national teleconsultation platform "Qare" in 4 common infections. 143,428 TCs with structured prescriptions were analyzed, with an appropriate prescription in more than 82% of cases, higher than in the literature. The use of data makes it possible to quickly assess practices and inform doctors to improve their practices.

Clinical Knowledge Platform (CKP): A Collaborative Ecosystem to Share Interoperable Clinical Forms, Viewers, and Order Sets with Various EMRs

A large number of Electronic Medical Records (EMR) are currently available with a variety of features and architectures. Existing studies and frameworks presented some solutions to overcome the problem of specification and application of clinical guidelines toward the automation of their use at the point of care. However, they could not yet support thoroughly the dynamic use of medical knowledge in EMRs according to the clinical contexts and provide local application of international recommendations. This study presents the development of the Clinical Knowledge Platform (CKP): a collaborative interoperable environment to create, use, and share sets of information elements that we entitled Clinical Use Contexts (CUCs). A CUC could include medical forms, patient dashboards, and order sets that are usable in various EMRs. For this purpose, we have identified and developed three basic requirements: an interoperable, inter-mapped dictionary of concepts leaning on standard terminologies, the possibility to define relevant clinical contexts, and an interface for collaborative content production via communities of professionals. Community members work together to create and/or modify, CUCs based on different clinical contexts. These CUCs will then be uploaded to be used in clinical applications in various EMRs. With this method, each CUC is, on the one hand, specific to a clinical context and on the other hand, could be adapted to the local practice conditions and constraints. Once a CUC has been developed, it could be shared with other potential users that can consume it directly or modify it according to their needs.

Digital Health Multilingual Ontology to Index Teaching Resources

The aim of this paper is to present the use of Medical Informatics Multilingual Ontology (MIMO) to index digital health resources that are (and will be) included in SaNuRN (project to teach digital health). MIMO currently contains 1,379 concepts and is integrated into HeTOP, which is a cross-lingual multiterminogy server. Existing teaching resources have been reindexed with MIMO concepts and integrated into a dedicated website. A total of 345 resources have been indexed with MIMO concepts and are freely available at https://doccismef.chu-rouen.fr/dc/#env=sanurn. The development of a multilingual MIMO for enhancing the quality and the efficiency of international projects is challenging. A specific semantic search engine has been deployed to give access to digital health teaching resources.

Reliability of citations of medRxiv preprints in articles published on COVID-19 in the world leading medical journals

Introduction

Preprints have been widely cited during the COVID-19 pandemics, even in the major medical journals. However, since subsequent publication of preprint is not always mentioned in preprint repositories, some may be inappropriately cited or quoted. Our objectives were to assess the reliability of preprint citations in articles on COVID-19, to the rate of publication of preprints cited in these articles and to compare, if relevant, the content of the preprints to their published version.

Methods

Articles published on COVID in 2020 in the BMJ, The Lancet, the JAMA and the NEJM were manually screened to identify all articles citing at least one preprint from medRxiv. We searched PubMed, Google and Google Scholar to assess if the preprint had been published in a peer-reviewed journal, and when. Published articles were screened to assess if the title, data or conclusions were identical to the preprint version.

Results

Among the 205 research articles on COVID published by the four major medical journals in 2020, 60 (29.3%) cited at least one medRxiv preprint. Among the 182 preprints cited, 124 were published in a peer-reviewed journal, with 51 (41.1%) before the citing article was published online and 73 (58.9%) later. There were differences in the title, the data or the conclusion between the preprint cited and the published version for nearly half of them. MedRxiv did not mentioned the publication for 53 (42.7%) of preprints.

Conclusions

More than a quarter of preprints citations were inappropriate since preprints were in fact already published at the time of publication of the citing article, often with a different content. Authors and editors should check the accuracy of the citations and of the quotations of preprints before publishing manuscripts that cite them.

Medical Informatics and Digital Health Multilingual Ontology (MIMO): a tool to improve international collaborations

Background

Even if English is the leading language for international communication, it is essential to keep in mind that research runs at the local level by local teams generally communicating in their local/national language, especially in Europe among European projects.

Objective

Therefore, the European Federation for Medical Informatics - Working Group on Health Informatics for Inter-regional Cooperation” has one objective: To develop a multilingual ontology focusing on Health Informatics and Digital Health as a collaboration tool that improves international and, in particular, European collaborations.

Results

We have developed the Medical Informatics and Digital Health Multilingual Ontology (MIMO). Hosted on the Health Terminology/Ontology Portal (HeTOP), MIMO contains around 1,000 concepts, 460 MeSH Descriptors, 220 MeSH Concepts, and more than 300 newly created concepts. MIMO is continuously updated to comprise as recent as possible concepts and their translations in more than 30 languages. Moreover, the MIMO’s development team constantly improves MIMO content and supporting information. Thus, during workshop discussions and one-on-one exchanges, the MIMO team has collected domain experts’ opinions about the community’s interests and suggestions for future enhancements. Moreover, MIMO will be integrated to support the annotation and categorization of research products into the HosmartAI European project involving more than 20 countries around Europe and worldwide.

Conclusion

MIMO is hosted by HeTOP (Health Terminology/Ontology Portal), which integrates 100 terminologies and ontologies in 55 languages. MIMO is freely available online. MIMO is portable to other knowledge platforms as part of MIMO’s main aims to facilitate communication between medical librarians, translators, and researchers as well as to support students’ self-learning.

Qualitative Evaluation of a Drug Terminology Server

BACKGROUND

Although the drug is finished, identifiable, there is no universally accepted standard for naming them. The objective of this work is to evaluate qualitatively the HeTOP drug terminology server by two categories of students: (a) pharmacy students and (b) a control group.

METHODS

A formal evaluation was built to measure the perception of users about the HeTOP drug server, using the three mains questions about "teaching interest", "skill interest" (or competence) and "ergonomics".

RESULTS

The three pharmacy student subgroups gave the best and the worst score to the same categories.

CONCLUSION

All three criteria are rated above 6.5 out of 10. The HeTOP drug terminology server is freely available to "non drug" specialists (URL: www.hetop.eu/hetop/drugs/).

Using Clinical Data Warehouse to Optimize the Vaccination Strategy Against COVID-19: A Use Case in France

Clinical Data Warehouses (CDW) are gold mines and may be useful to manage the COVID-19 outbreak. This article details the use of CDW in order to retrieve patients for vaccination purposes. A list of 34 diseases (or conditions) was published by French Health Authorities to target individuals at a high risk of developing a severe form of COVID. Using a multilevel search engine, 23 queries were built based on structured or unstructured data using natural language processing features. The Diagnosis Related Group coding system was used alone in three queries (13.0%), coupled with unstructured data in four queries (17.4%), and unstructured data were used alone in 16 queries (69.6%). Eleven diseases (conditions) were too broad to be translated into queries. Finally, 6,006 unique re-identified patients were retrieved. This use case demonstrates the usefulness of the Rouen University Hospital CDW in retrieving patients for other purposes than translational research.

OntoBioStat: Supporting Causal Diagram Design and Analysis

Suitable causal inference in biostatistics can be best achieved by knowledge representation thanks to causal diagrams or directed acyclic graphs. However, necessary and sufficient causes are not easily represented. Since existing ontologies do not fill this gap, we designed OntoBioStat in order to enable covariate selection support based on causal relation representations. OntoBioStat automatic ontological causal diagram construction and inferences are detailed in this study. OntoBioStat inferences are allowed by Semantic Web Rule Language rules and axioms. First, statements made by the users include outcome, exposure, covariate, and causal relation specification. Then, reasoning enable automatic construction using generic instances of Meta_Variable and Necessary_Variable classes. Finally, inferred classes highlighted potential bias such as confounder-like. Ontological causal diagram built with OntoBioStat was compared to a standard causal diagram (without OntoBioStat) in a theoretical study. It was found that confounding and bias were not completely identified by the standard causal diagram, and erroneous covariate sets were provided. Further research is needed in order to make OntoBioStat more usable.

Reliability of Drug-Drug Interaction Measurement on Real-Word Data: The ReMIAMes Project

The ReMIAMes project proposes a methodological framework to provide a reliable and reproducible measurement of the frequency of drug-drug interactions (DDI) when performed on real-world data. This framework relies on (i) a fine-grained and contextualized definition of DDIs, (ii) a shared minimum information model to select the appropriate data for the correct interpretation of potential DDIs, (iii) an ontology-based inference module able to handle missing data to classify prescription lines with potential DDIs, (iv) a report generator giving the value of the measurement and explanations when potential false positive are detected due to a lack of available data. All the tools developed are intended to be publicly shared under open license.

When Context Matters for Credible Measurement of Drug-Drug Interactions Based on Real-World Data

The frequency of potential drug-drug interactions (DDI) in published studies on real world data considerably varies due to the methodological framework. Contextualization of DDI has a proven effect in limiting false positives. In this paper, we experimented with the application of various DDIs contexts elements to see their impact on the frequency of potential DDIs measured on the same set of prescription data collected in EDSaN, the clinical data warehouse of Rouen University Hospital. Depending on the context applied, the frequency of daily prescriptions with potential DDI ranged from 0.89% to 3.90%. Substance-level analysis accounted for 48% of false positives because it did not account for some drug-related attributes. Consideration of the patient's context could eliminate up to an additional 29% of false positives.

WikiMeSH: Multi Lingual MeSH Translations via Wikipedia

OBJECTIVE

The aim of this paper is to propose an extended translation of the MeSH thesaurus based on Wikipedia pages.

METHODS

A mapping was realized between each MeSH descriptor (preferred terms and synonyms) and corresponding Wikipedia pages.

RESULTS

A tool called "WikiMeSH" has been developed. Among the top 20 languages of this study, seven have currently no MeSH translations: Arabic, Catalan, Farsi (Iran), Mandarin Chinese, Korean, Serbian, and Ukrainian. For these seven languages, WikiMeSH is proposing a translation for 47% for Arabic to 34% for Serbian.

CONCLUSION

WikiMeSH is an interesting tool to translate the MeSH thesaurus and other health terminologies and ontologies based on a mapping to Wikipedia pages.

Delayed gastric emptying as an independent predictor of mortality in gastroparesis

BACKGROUND

Whether gastroparesis is associated with a shortened life expectancy remains uncertain as no systematic study has evaluated the impact of gastroparesis on mortality, based on gastric emptying (GE) tests.

AIM

This study aimed to assess whether delayed GE was predictive of mortality.

METHODS

GE was measured using a 13C-octanoic acid breath test in 1563 consecutive patients. Delayed GE at baseline defined the gastroparesis group. Patients were followed up for a mean of 8.9 years, yielding 13 466 patients per year. Mortality was assessed using the French CepiDc database with data from local civil registries. The cause of death was determined from medical records. Mortality rates were assessed using the Kaplan-Meier method and hazard ratio (HR) was calculated using the Cox regression model.

RESULTS

Age and symptoms severity were not different among patients with normal GE (n = 1179) and with delayed GE (n = 384) while diabetes mellitus was more frequent in the gastroparesis group. Kaplan-Meier analysis showed increased mortality in the gastroparesis group compared to patients with normal GE. Cox regression model identified delayed GE as independently associated with increased mortality (HR = 1.63[1.09-2.42]; P = 0.02). Other independent factors associated with increased mortality included age, male sex, and diabetes. No difference was observed between groups for the cause of death, with cancer and cardiovascular disease being the leading causes.

CONCLUSION

This study has shown that gastroparesis, diagnosed on GE tests, was associated with increased mortality, independently of age, sex, BMI or diabetes status (NCT04918329).

Designing Formulae for Ranking Search Results: Mixed Methods Evaluation Study

Background

A major factor in the success of any search engine is the relevance of the search results; a tool should sort the search results to present the most relevant documents first. Assessing the performance of the ranking formula is an important part of search engine evaluation. However, the methods currently used to evaluate ranking formulae mainly collect quantitative data and do not gather qualitative data, which help to understand what needs to be improved to tailor the formulae to their end users.

Objective

This study aims to evaluate 2 different parameter settings of the ranking formula of LiSSa (the French acronym for scientific literature in health care; Department of Medical Informatics and Information), a tool that provides access to health scientific literature in French, to adapt the formula to the needs of the end users.

Methods

To collect quantitative and qualitative data, user tests were carried out with representative end users of LiSSa: 10 general practitioners and 10 registrars. Participants first assessed the relevance of the search results and then rated the ranking criteria used in the 2 formulae. Verbalizations were analyzed to characterize each criterion.

Results

A formula that prioritized articles representing a consensus in the field was preferred. When users assess an article’s relevance, they judge its topic, methods, and value in clinical practice.

Conclusions

Following the evaluation, several improvements were implemented to give more weight to articles that match the search topic and to downgrade articles that have less informative or scientific value for the reader. Applying a qualitative methodology generates valuable user inputs to improve the ranking formula and move toward a highly usable search engine.

Evaluation of Doc'EDS: a French semantic search tool to query health documents from a clinical data warehouse

BACKGROUND

Unstructured data from electronic health records represent a wealth of information. Doc'EDS is a pre-screening tool based on textual and semantic analysis. The Doc'EDS system provides a graphic user interface to search documents in French. The aim of this study was to present the Doc'EDS tool and to provide a formal evaluation of its semantic features.

METHODS

Doc'EDS is a search tool built on top of the clinical data warehouse developed at Rouen University Hospital. This tool is a multilevel search engine combining structured and unstructured data. It also provides basic analytical features and semantic utilities. A formal evaluation was conducted to measure the impact of Natural Language Processing algorithms.

RESULTS

Approximately 18.1 million narrative documents are stored in Doc'EDS. The formal evaluation was conducted in 5000 clinical concepts that were manually collected. The F-measures of negative concepts and hypothetical concepts were respectively 0.89 and 0.57.

CONCLUSION

In this formal evaluation, we have shown that Doc'EDS is able to deal with language subtleties to enhance an advanced full text search in French health documents. The Doc'EDS tool is currently used on a daily basis to help researchers to identify patient cohorts thanks to unstructured data.

Assisting Data Retrieval with a Drug Knowledge Graph

The Normandy health data warehouse EDSaN integrates the medication orders from the University Hospital of Rouen (France). This study aims at describing the design and the evaluation of an information retrieval system founded on a complex and semantically augmented knowledge graph dedicated to EDSaN drugs' prescriptions. The system is intended to help the selection of drugs in the search process by health professionals. The manual evaluation of the relevance of the returned drugs showed encouraging results as expected. A deeper analysis in order to improve the ranking method is needed and will be performed in a future work.

Telemedicine in French Memory Clinics During the COVID-19 Pandemic

This multicenter study was conducted in French memory clinics during the first COVID-2019 lockdown (March-May 2020). The objective was to evaluate the effect of a telemedicine consultation on treatment modification in dementia care. Among 874 patients who had a telemedicine consultation, 103 (10.7%) had treatment modifications, in particular those living with a relative or diagnosed with Alzheimer's disease. A control group of patients referred March-May 2019 was also included. Treatment modification rate was similar between periods with an adjusted percentage difference of -4% (p = 0.27). Telemedicine consultations allowed treatment modifications with only a minor short-term negative impact on therapeutic strategies.

Telemedicine in French memory clinics during Covid‐19 crisis

Background

In early 2020, COVID‐19 outbreak struck France leading to a national lockdown between March 17^th and May 11^th. While standard in‐person medical consultation was complicated, telemedicine dramatically expanded. In order to evaluate the impact of this unpreceded situation on clinical practice and use of psychoactive drug in dementia care, we conducted a nationwide clinical prospective and retrospective study.

Method

During the lockdown period, telemedicine patients’ demographic and clinical data were retrospectively collected from 7 French memory clinics (telemedicine cohort). Clinical diagnoses, treatment changes, cognitive modifications since last consultations and living conditions during the lockdown were systematically retrieved.

In Rouen site, we also included patients only reached by a secretary to propose a postponed visit after lockdown (no‐telemedicine cohort) and patients seen in 2019 during the same period of the year (Rouen‐2019).

The primary outcome was any change in psychoactive drug and a specific analysis on sedative treatment increase was the secondary outcome, defined as any increase in the prescriptions of antipsychotics or benzodiazepines.

Result

The telemedicine cohort included 874 patients (73 from Rouen), while no‐telemedicine control cohort and Rouen‐2019 cohorts included respectively 86 and 234 patients (table 1). In the telemedicine cohort, treatments were modified for 10.7% of the patients with more treatment modification among the patients living with a relative (+5.8% (CI95% [0.2%; 11.4%] p=0.04) and among the patients with Alzheimer’s disease (+12.2% (CI95% [7.1%; 17.3%] p<0.001). When comparing therapeutic strategies in 2020 and 2019 for Rouen site, 24.6% of the patients had their treatment modified in 2020 and 12.4% in 2019. That difference was however not statically significant with an adjusted percentage difference of ‐4% (CI95% [‐10.8%; 3.4%] p=0.27, including the telemedicine and no‐telemedicine cohorts for 2020.

Conclusion

Telemedicine seems to have had only minor negative impacts on clinical practice in memory clinics.

Immunomodulatory or/and immunosuppressive drugs should not avoid skin test for the assessment of drug allergy

The use of immunomodulatory and/or immunosuppressive therapy (IT) is increasingly common in the management of chronic inflammatory disease. Skin reactions to any drug (IT or not) are not rare in these patients, justifying allergological investigations. The influence of IT on allergological tests for drugs is not clearly described. IT cannot be interrupted due to the underlying disease. The data assessing the benefit and the safety of allergological test for drug allergy in patients under IT are missing.

Ontological Models Supporting Covariates Selection in Observational Studies

In the context of causal inference, biostatisticians use causal diagrams to select covariates in order to build multivariate models. These diagrams represent datasets variables and their relations but have some limitations (representing interactions, bidirectional causal relations). The MetBrAYN project aims at building an ontological-based process to tackle these issues. The knowledge acquired by the biostatistician during a methodological consultation for a research question will be represented in a general ontology. In order to aggregate various forms of knowledge the ontology will act as a wrapper. Ontology-based causal diagrams will be semi-automatically built. Founded on inference rules, the global system will help biostatisticians to curate it and to visualize recommended covariates for their research question.

Analysis of Publication Trends in Childhood Obesity Research in PubMed Since 1945

Background: Childhood obesity (CO) has become a true epidemic and a subject of increasing publications. The aim of this study was to assess if the number of publications in that field increases over time in proportion to the epidemic, and also according to socioeconomic factors. Methods: A PubMed search was carried out to extract articles related to CO published between 1945 and 2017. Data were downloaded from PubMed and processed through a dedicated parser. Socioeconomic data were collected from international organizations. Results: Overall, 36,554 articles were retrieved among 3329 journals, one-third of them being concentrated in 44 journals. The annual growth rate of publications on CO was on average 11.6% per year between 1990 and 2016, whereas the growth rate of articles on pediatrics or of the total articles indexed in MEDLINE was 2.6% and 4.4%, respectively. The most productive countries were the United States (37.80%), the United Kingdom (6.24%), and Italy (4.56%). There was a significant relationship between publications on CO in a country and prevalence of CO in that country (p = 0.002) and between evolution of the number of publications and evolution of the Human Development Index (p = 0.01). Following exponential growth, CO publications reached a plateau in 2013, whereas publications targeted on obesity in infants continue to increase. Conclusions: Research on CO has risen markedly in the last two decades, with a higher growth rate than biomedical research overall, as a result of the worldwide obesity epidemic and also due to specific socioeconomic factors.

Manual Evaluation of the Automatic Mapping of International Classification of Diseases (ICD)-11 (in French)

A lexical method was used to map ICD-11 to the terminologies included in the HeTOP server. About half of ICD-11 codes (47.76%) were mapped to at least one concept. The developed tool reached a global precision of 0.98 and a recall of 0.66. Lexical methods are powerful methods to map health terminologies. Supervised and manual mapping is still necessary to complete the mapping.

Identification of the Best Semantic Expansion to Query PubMed Through Automatic Performance Assessment of Four Search Strategies on All Medical Subject Heading Descriptors: Comparative Study

Background

With the continuous expansion of available biomedical data, efficient and effective information retrieval has become of utmost importance. Semantic expansion of queries using synonyms may improve information retrieval.

Objective

The aim of this study was to automatically construct and evaluate expanded PubMed queries of the form “preferred term”[MH] OR “preferred term”[TIAB] OR “synonym 1”[TIAB] OR “synonym 2”[TIAB] OR …, for each of the 28,313 Medical Subject Heading (MeSH) descriptors, by using different semantic expansion strategies. We sought to propose an innovative method that could automatically evaluate these strategies, based on the three main metrics used in information science (precision, recall, and F-measure).

Methods

Three semantic expansion strategies were assessed. They differed by the synonyms used to build the queries as follows: MeSH synonyms, Unified Medical Language System (UMLS) mappings, and custom mappings (Catalogue et Index des Sites Médicaux de langue Française [CISMeF]). The precision, recall, and F-measure metrics were automatically computed for the three strategies and for the standard automatic term mapping (ATM) of PubMed. The method to automatically compute the metrics involved computing the number of all relevant citations (A), using National Library of Medicine indexing as the gold standard (“preferred term”[MH]), the number of citations retrieved by the added terms (”synonym 1“[TIAB] OR ”synonym 2“[TIAB] OR …) (B), and the number of relevant citations retrieved by the added terms (combining the previous two queries with an “AND” operator) (C). It was possible to programmatically compute the metrics for each strategy using each of the 28,313 MeSH descriptors as a “preferred term,” corresponding to 239,724 different queries built and sent to the PubMed application program interface. The four search strategies were ranked and compared for each metric.

Results

ATM had the worst performance for all three metrics among the four strategies. The MeSH strategy had the best mean precision (51%, SD 23%). The UMLS strategy had the best recall and F-measure (41%, SD 31% and 36%, SD 24%, respectively). CISMeF had the second best recall and F-measure (40%, SD 31% and 35%, SD 24%, respectively). However, considering a cutoff of 5%, CISMeF had better precision than UMLS for 1180 descriptors, better recall for 793 descriptors, and better F-measure for 678 descriptors.

Conclusions

This study highlights the importance of using semantic expansion strategies to improve information retrieval. However, the performances of a given strategy, relatively to another, varied greatly depending on the MeSH descriptor. These results confirm there is no ideal search strategy for all descriptors. Different semantic expansions should be used depending on the descriptor and the user’s objectives. Thus, we developed an interface that allows users to input a descriptor and then proposes the best semantic expansion to maximize the three main metrics (precision, recall, and F-measure).

Building a Semantic Health Data Warehouse in the Context of Clinical Trials: Development and Usability Study

Background

The huge amount of clinical, administrative, and demographic data recorded and maintained by hospitals can be consistently aggregated into health data warehouses with a uniform data model. In 2017, Rouen University Hospital (RUH) initiated the design of a semantic health data warehouse enabling both semantic description and retrieval of health information.

Objective

This study aimed to present a proof of concept of this semantic health data warehouse, based on the data of 250,000 patients from RUH, and to assess its ability to assist health professionals in prescreening eligible patients in a clinical trials context.

Methods

The semantic health data warehouse relies on 3 distinct semantic layers: (1) a terminology and ontology portal, (2) a semantic annotator, and (3) a semantic search engine and NoSQL (not only structured query language) layer to enhance data access performances. The system adopts an entity-centered vision that provides generic search capabilities able to express data requirements in terms of the whole set of interconnected conceptual entities that compose health information.

Results

We assessed the ability of the system to assist the search for 95 inclusion and exclusion criteria originating from 5 randomly chosen clinical trials from RUH. The system succeeded in fully automating 39% (29/74) of the criteria and was efficiently used as a prescreening tool for 73% (54/74) of them. Furthermore, the targeted sources of information and the search engine–related or data-related limitations that could explain the results for each criterion were also observed.

Conclusions

The entity-centered vision contrasts with the usual patient-centered vision adopted by existing systems. It enables more genericity in the information retrieval process. It also allows to fully exploit the semantic description of health information. Despite their semantic annotation, searching within clinical narratives remained the major challenge of the system. A finer annotation of the clinical texts and the addition of specific functionalities would significantly improve the results. The semantic aspect of the system combined with its generic entity-centered vision enables the processing of a large range of clinical questions. However, an important part of health information remains in clinical narratives, and we are currently investigating novel approaches (deep learning) to enhance the semantic annotation of those unstructured data.

Evaluation of automatic annotation by a multi-terminological concepts extractor within a corpus of data from family medicine consultations

INTRODUCTION

Research in family medicine is necessary to improve the quality of care. The number of publications in general medicine remains low. Databases from Electronic Medical Records can increase the number of these publications. These data must be coded to be used pertinently. The objective of this study was to assess the quality of semantic annotation by a multi-terminological concept extractor within a corpus of family medicine consultations.

METHOD

Consultation data in French from 25 general practitioners were automatically annotated using 28 different terminologies. The data extracted were classified into three groups: reasons for consulting, observations and consultation results. The first evaluation led to a correction phase of the tool which led to a second evaluation. For each evaluation, the precision, recall and F-measure were quantified. Then, the inter- and intra-terminological coverage of each terminology was assessed.

RESULTS

Nearly 15,000 automatic annotations were manually evaluated. The mean values for the second evaluation of precision, recall and F-measure were 0.85, 0.83 and 0.84 respectively. The most common terminologies used were SNOMED CT, SNOMED 3.5 and NClt. The terminologies with the best intra-terminological coverage were ICPC-2, DRC and CISMeF Meta-Terms.

CONCLUSION

A multi-terminological concepts extractor can be used for the automatic annotation of consultation data in family medicine. Integrating such a tool into general practitioners' business software would be a solution to the lack of routine coding. Developing the use of a single terminology specific to family medicine could improve coding, facilitate semantic interoperability and the communication of relevant information.

ApiAppS: A Project to Study and Help Practitioners in Recommending mHealth Apps and Devices to Their Patients

The ApiAppS ongoing project aims to provide physicians with a decision support system for the prescription / recommendation of mHealth technologies. We describe the context and the components of the project which includes: 1) a technical part on modelling and implementing the decision support system, and 2) a psychosocial investigation part designed to have a better knowledge of general practitioners (GPs) and patients' expectations, beliefs and practices.

Word Embedding for French Natural Language in Healthcare: A Comparative Study

Structuring raw medical documents with ontology mapping is now the next step for medical intelligence. Deep learning models take as input mathematically embedded information, such as encoded texts. To do so, word embedding methods can represent every word from a text as a fixed-length vector. A formal evaluation of three word embedding methods has been performed on raw medical documents. The data corresponds to more than 12M diverse documents produced in the Rouen hospital (drug prescriptions, discharge and surgery summaries, inter-services letters, etc.). Automatic and manual validation demonstrates that Word2Vec based on the skip-gram architecture had the best rate on three out of four accuracy tests. This model will now be used as the first layer of an AI-based semantic annotator.

Word Embedding for the French Natural Language in Health Care: Comparative Study

Background

Word embedding technologies, a set of language modeling and feature learning techniques in natural language processing (NLP), are now used in a wide range of applications. However, no formal evaluation and comparison have been made on the ability of each of the 3 current most famous unsupervised implementations (Word2Vec, GloVe, and FastText) to keep track of the semantic similarities existing between words, when trained on the same dataset.

Objective

The aim of this study was to compare embedding methods trained on a corpus of French health-related documents produced in a professional context. The best method will then help us develop a new semantic annotator.

Methods

Unsupervised embedding models have been trained on 641,279 documents originating from the Rouen University Hospital. These data are not structured and cover a wide range of documents produced in a clinical setting (discharge summary, procedure reports, and prescriptions). In total, 4 rated evaluation tasks were defined (cosine similarity, odd one, analogy-based operations, and human formal evaluation) and applied on each model, as well as embedding visualization.

Results

Word2Vec had the highest score on 3 out of 4 rated tasks (analogy-based operations, odd one similarity, and human validation), particularly regarding the skip-gram architecture.

Conclusions

Although this implementation had the best rate for semantic properties conservation, each model has its own qualities and defects, such as the training time, which is very short for GloVe, or morphological similarity conservation observed with FastText. Models and test sets produced by this study will be the first to be publicly available through a graphical interface to help advance the French biomedical research.

[An online dynamic knowledge base in multiple languages on general medicine and primary care]

INTRODUCTION

The International Classification of Primary Care, Second version (ICPC-2) aligned with the 10th Revision of the International Classification of Disease (ICD-10) is a standard for primary care epidemiology compendium. ICPC-2 has been also intended to identify the clinical topics in family medicine. Contextual field-specific knowledge in family medicine and primary care such as health structures, management, categories of patients, research methods, ethical or environmental features are not standardized and reflect, more often, the views of experts.

METHODS

A qualitative research method, applied to the analysis of several Family Medicine congresses, has helped identify, in addition to clinical items, a spectrum of contextual concepts addressed by family doctors during their exchanges at the congresses. Assembled in a hierarchical manner, these concepts were given expression, together with ICPC-2, under the name of Q-codes Version 2.5, in the multilingual multi-terminology semantic server of the Department of Information and medical informatics (D2Im) at the University of Rouen, France. The two classifications are edited under the acronym 3 CGP for Core Content classification of General Practice. This free access server allows you to consult the ICPC-2 in 22 languages and the Q-codes in ten languages.

RESULTS

The result of the joint use of these two classifications, as descriptors in congress to identify the concepts in texts or index the gray literature for family medicine and primary care is presented here in its various pilot uses. The validity and generalizability of 3CGP appears to be good in the light of the translations already carried out by colleagues around the world and of the applicability of the method in the two sides of the Atlantic. However the reproducibility and the inter-coder variations still remain to be tested for Q-codes. Maintenance remains an issue.

CONCLUSION

This method highlights the conceptual extension, the complexity and the dynamics of the role of general practitioner and family doctor as well as of primary care physician.

Development, dissemination, and applications of a new terminological resource, the Q-Code taxonomy for professional aspects of general practice/family medicine

BACKGROUND

While documentation of clinical aspects of General Practice/Family Medicine (GP/FM) is assured by the International Classification of Primary Care (ICPC), there is no taxonomy for the professional aspects (context and management) of GP/FM.

OBJECTIVES

To present the development, dissemination, applications, and resulting face validity of the Q-Codes taxonomy specifically designed to describe contextual features of GP/FM, proposed as an extension to the ICPC.

DEVELOPMENT

The Q-Codes taxonomy was developed from Lamberts' seminal idea for indexing contextual content (1987) by a multi-disciplinary team of knowledge engineers, linguists and general practitioners, through a qualitative and iterative analysis of 1702 abstracts from six GP/FM conferences using Atlas.ti software. A total of 182 concepts, called Q-Codes, representing professional aspects of GP/FM were identified and organized in a taxonomy. Dissemination: The taxonomy is published as an online terminological resource, using semantic web techniques and web ontology language (OWL) ( http://www.hetop.eu/Q ). Each Q-Code is identified with a unique resource identifier (URI), and provided with preferred terms, and scope notes in ten languages (Portuguese, Spanish, English, French, Dutch, Korean, Vietnamese, Turkish, Georgian, German) and search filters for MEDLINE and web searches.

APPLICATIONS

This taxonomy has already been used to support queries in bibliographic databases (e.g., MEDLINE), to facilitate indexing of grey literature in GP/FM as congress abstracts, master theses, websites and as an educational tool in vocational teaching, Conclusions: The rapidly growing list of practical applications provides face-validity for the usefulness of this freely available new terminological resource.