Digital Health Education for the Future: The SaNuRN (Santé Numérique Rouen-Nice) Consortium's Journey

SaNuRN is a five-year project by the University of Rouen Normandy (URN) and the Côte d’Azur University (CAU) consortium to optimize digital health education for medical and paramedical students, professionals, and administrators. The project includes a skills framework, training modules, and teaching resources. In 2027, SaNuRN is expected to train a significant portion of the 400,000 health and paramedical professions students at the French national level. Our purpose is to give a synopsis of the SaNuRN initiative, emphasizing its novel educational methods and how they will enhance the delivery of digital health education. Our goals include showcasing SaNuRN as a comprehensive program consisting of a proficiency framework, instructional modules, and educational materials and explaining how SaNuRN is implemented in the participating academic institutions. SaNuRN is a project aimed at educating and training health-related and paramedics students in digital health. The project results from a cooperative effort between URN and CAU, covering four French departments. The project is based on the French National Referential on Digital Health (FNRDH), which defines the skills and competencies to be acquired and validated by every student in the health, paramedical, and social professions curricula. The SaNuRN team is currently adapting the existing URN and CAU syllabi to FNRDH and developing short-duration video capsules of 20 to 30 minutes to teach all the relevant material. The project aims to ensure that the largest student population earns the necessary skills, and it has developed a two-tier system involving facilitators who will enable the efficient expansion of the project’s educational outreach and support the students in learning the needed material efficiently. With a focus on real-world scenarios and innovative teaching activities integrating telemedicine devices and virtual professionals, SaNuRN is committed to enabling continuous learning for healthcare professionals in clinical practice. The SaNuRN team introduced new ways of evaluating healthcare professionals by shifting from a knowledge-based to a competencies-based evaluation, aligning with the Miller teaching pyramid and using the Objective Structured Clinical Examination and Script Concordance Test in digital health education. Drawing on the expertise of URN, CAU, and their public health and digital research laboratories and partners, the SaNuRN project represents a platform for continuous innovation, including telemedicine training and living labs with virtual and interactive professional activities. The SaNuRN project provides a comprehensive, personalized 30-hour training package for health and paramedical students, addressing all 70 FNRDH competencies. The program is enhanced using AI and NLP to create virtual patients and professionals for digital healthcare simulation. SaNuRN teaching materials are open-access. The project collaborates with academic institutions worldwide to develop educational material in digital health in English and multilingual formats. SaNuRN offers a practical and persuasive training approach to meet the current digital health education requirements.

Sharing Digital Health Educational Resources in a One-Stop Shop Portal: Tutorial on the Catalog and Index of Digital Health Teaching Resources (CIDHR) Semantic Search Engine

BACKGROUND

Access to reliable and accurate digital health web-based resources is crucial. However, the lack of dedicated search engines for non-English languages, such as French, is a significant obstacle in this field. Thus, we developed and implemented a multilingual, multiterminology semantic search engine called Catalog and Index of Digital Health Teaching Resources (CIDHR). CIDHR is freely accessible to everyone, with a focus on French-speaking resources. CIDHR has been initiated to provide validated, high-quality content tailored to the specific needs of each user profile, be it students or professionals.

OBJECTIVE

This study's primary aim in developing and implementing the CIDHR is to improve knowledge sharing and spreading in digital health and health informatics and expand the health-related educational community, primarily French speaking but also in other languages. We intend to support the continuous development of initial (ie, bachelor level), advanced (ie, master and doctoral levels), and continuing training (ie, professionals and postgraduate levels) in digital health for health and social work fields. The main objective is to describe the development and implementation of CIDHR. The hypothesis guiding this research is that controlled vocabularies dedicated to medical informatics and digital health, such as the Medical Informatics Multilingual Ontology (MIMO) and the concepts structuring the French National Referential on Digital Health (FNRDH), to index digital health teaching and learning resources, are effectively increasing the availability and accessibility of these resources to medical students and other health care professionals.

METHODS

First, resource identification is processed by medical librarians from websites and scientific sources preselected and validated by domain experts and surveyed every week. Then, based on MIMO and FNRDH, the educational resources are indexed for each related knowledge domain. The same resources are also tagged with relevant academic and professional experience levels. Afterward, the indexed resources are shared with the digital health teaching and learning community. The last step consists of assessing CIDHR by obtaining informal feedback from users.

RESULTS

Resource identification and evaluation processes were executed by a dedicated team of medical librarians, aiming to collect and curate an extensive collection of digital health teaching and learning resources. The resources that successfully passed the evaluation process were promptly included in CIDHR. These resources were diligently indexed (with MIMO and FNRDH) and tagged for the study field and degree level. By October 2023, a total of 371 indexed resources were available on a dedicated portal.

CONCLUSIONS

CIDHR is a multilingual digital health education semantic search engine and platform that aims to increase the accessibility of educational resources to the broader health care-related community. It focuses on making resources "findable," "accessible," "interoperable," and "reusable" by using a one-stop shop portal approach. CIDHR has and will have an essential role in increasing digital health literacy.

WikiMeSH: Multi Lingual MeSH Translations via Wikipedia

OBJECTIVE

The aim of this paper is to propose an extended translation of the MeSH thesaurus based on Wikipedia pages.

METHODS

A mapping was realized between each MeSH descriptor (preferred terms and synonyms) and corresponding Wikipedia pages.

RESULTS

A tool called "WikiMeSH" has been developed. Among the top 20 languages of this study, seven have currently no MeSH translations: Arabic, Catalan, Farsi (Iran), Mandarin Chinese, Korean, Serbian, and Ukrainian. For these seven languages, WikiMeSH is proposing a translation for 47% for Arabic to 34% for Serbian.

CONCLUSION

WikiMeSH is an interesting tool to translate the MeSH thesaurus and other health terminologies and ontologies based on a mapping to Wikipedia pages.

Physician satisfaction with transition from CPOE to paper-based prescription

INTRODUCTION

In January 2015, Rouen University Hospital's information system experienced serious issues. It was necessary to rapidly switch from the computerized provider order entry (CPOE) system towards a paper-based order entry (PBOE) system. This was an opportunity to evaluate prescriber opinion on the two provider order entry (POE) systems.

METHODS

All residents were asked to fill an augmented version of the POE satisfaction and usage survey for both POE systems. The results were compared to identify the strengths and weaknesses of each system.

RESULTS

Fifty-one respondents had used the CPOE system and the PBOE system. Overall, satisfaction was higher with PBOE than CPOE (odds ratio (OR)=3.74; p<0.001). Usability (OR=4.00; p<0.001), reliability (OR=8.54; p<0.001), time consumption (OR=0.50; p<0.05 - survey statement was formulated negatively), and communication with nurses (OR=14.27; p<0.0001) reached statistically better agreement. The more experience with CPOE the more residents were disillusioned with the reliability (OR=6.55; p<0.01), the usability (OR=5.68; p<0.01) and the patient safety (OR=0.27; p<0.05 - survey statement was formulated negatively) of CPOE. Although safety issues were reported for both systems, the causes were different; PBOE imposed frequent rewriting of the order while CPOE lack of usability might be unsafe. Another important issue with both POE systems was time consumption.

CONCLUSION

Residents did not report any increase in safety issues with the rapid switch from CPOE to PBOE. They even seemed more satisfied with the rollback to paper, which remains a possible degraded mode in case of health information technology collapse.

Searching for rare diseases in PubMed: a blind comparison of Orphanet expert query and query based on terminological knowledge

BACKGROUND

Despite international initiatives like Orphanet, it remains difficult to find up-to-date information about rare diseases. The aim of this study is to propose an exhaustive set of queries for PubMed based on terminological knowledge and to evaluate it versus the queries based on expertise provided by the most frequently used resource in Europe: Orphanet.

METHODS

Four rare disease terminologies (MeSH, OMIM, HPO and HRDO) were manually mapped to each other permitting the automatic creation of expended terminological queries for rare diseases. For 30 rare diseases, 30 citations retrieved by Orphanet expert query and/or query based on terminological knowledge were assessed for relevance by two independent reviewers unaware of the query's origin. An adjudication procedure was used to resolve any discrepancy. Precision, relative recall and F-measure were all computed.

RESULTS

For each Orphanet rare disease (n = 8982), there was a corresponding terminological query, in contrast with only 2284 queries provided by Orphanet. Only 553 citations were evaluated due to queries with 0 or only a few hits. There were no significant differences between the Orpha query and terminological query in terms of precision, respectively 0.61 vs 0.52 (p = 0.13). Nevertheless, terminological queries retrieved more citations more often than Orpha queries (0.57 vs. 0.33; p = 0.01). Interestingly, Orpha queries seemed to retrieve older citations than terminological queries (p < 0.0001).

CONCLUSION

The terminological queries proposed in this study are now currently available for all rare diseases. They may be a useful tool for both precision or recall oriented literature search.

Knowledge Representation and Management. From Ontology to Annotation. Findings from the Yearbook 2015 Section on Knowledge Representation and Management

OBJECTIVE

To summarize the best papers in the field of Knowledge Representation and Management (KRM).

METHODS

A comprehensive review of medical informatics literature was performed to select some of the most interesting papers of KRM published in 2014.

RESULTS

Four articles were selected, two focused on annotation and information retrieval using an ontology. The two others focused mainly on ontologies, one dealing with the usage of a temporal ontology in order to analyze the content of narrative document, one describing a methodology for building multilingual ontologies.

CONCLUSION

Semantic models began to show their efficiency, coupled with annotation tools.

[Limiting a Medline/PubMed query to the "best" articles using the JCR relative impact factor]

BACKGROUND

Medline/PubMed is the most frequently used medical bibliographic research database. The aim of this study was to propose a new generic method to limit any Medline/PubMed query based on the relative impact factor and the A & B categories of the SIGAPS score.

MATERIAL AND METHODS

The entire PubMed corpus was used for the feasibility study, then ten frequent diseases in terms of PubMed indexing and the citations of four Nobel prize winners. The relative impact factor (RIF) was calculated by medical specialty defined in Journal Citation Reports. The two queries, which included all the journals in category A (or A OR B), were added to any Medline/PubMed query as a central point of the feasibility study.

RESULTS

Limitation using the SIGAPS category A was larger than the when using the Core Clinical Journals (CCJ): 15.65% of PubMed corpus vs 8.64% for CCJ. The response time of this limit applied to the entire PubMed corpus was less than two seconds. For five diseases out of ten, limiting the citations with the RIF was more effective than with the CCJ. For the four Nobel prize winners, limiting the citations with the RIF was more effective than the CCJ.

CONCLUSION

The feasibility study to apply a new filter based on the relative impact factor on any Medline/PubMed query was positive.

A search engine to access PubMed monolingual subsets: proof of concept and evaluation in French

Background

PubMed contains numerous articles in languages other than English. However, existing solutions to access these articles in the language in which they were written remain unconvincing.

Objective

The aim of this study was to propose a practical search engine, called Multilingual PubMed, which will permit access to a PubMed subset in 1 language and to evaluate the precision and coverage for the French version (Multilingual PubMed-French).

Methods

To create this tool, translations of MeSH were enriched (eg, adding synonyms and translations in French) and integrated into a terminology portal. PubMed subsets in several European languages were also added to our database using a dedicated parser. The response time for the generic semantic search engine was evaluated for simple queries. BabelMeSH, Multilingual PubMed-French, and 3 different PubMed strategies were compared by searching for literature in French. Precision and coverage were measured for 20 randomly selected queries. The results were evaluated as relevant to title and abstract, the evaluator being blind to search strategy.

Results

More than 650,000 PubMed citations in French were integrated into the Multilingual PubMed-French information system. The response times were all below the threshold defined for usability (2 seconds). Two search strategies (Multilingual PubMed-French and 1 PubMed strategy) showed high precision (0.93 and 0.97, respectively), but coverage was 4 times higher for Multilingual PubMed-French.

Conclusions

It is now possible to freely access biomedical literature using a practical search tool in French. This tool will be of particular interest for health professionals and other end users who do not read or query sufficiently in English. The information system is theoretically well suited to expand the approach to other European languages, such as German, Spanish, Norwegian, and Portuguese.

Performance evaluation of Unified Medical Language System®'s synonyms expansion to query PubMed

BACKGROUND

PubMed is the main access to medical literature on the Internet. In order to enhance the performance of its information retrieval tools, primarily non-indexed citations, the authors propose a method: expanding users' queries using Unified Medical Language System' (UMLS) synonyms i.e. all the terms gathered under one unique Concept Unique Identifier.

METHODS

This method was evaluated using queries constructed to emphasize the differences between this new method and the current PubMed automatic term mapping. Four experts assessed citation relevance.

RESULTS

Using UMLS, we were able to retrieve new citations in 45.5% of queries, which implies a small increase in recall. The new strategy led to a heterogeneous 23.7% mean increase in non-indexed citation retrieved. Of these, 82% have been published less than 4 months earlier. The overall mean precision was 48.4% but differed according to the evaluators, ranging from 36.7% to 88.1% (Inter rater agreement was poor: kappa = 0.34).

CONCLUSIONS

This study highlights the need for specific search tools for each type of user and use-cases. The proposed strategy may be useful to retrieve recent scientific advancement.

[Searching French institutional health information sources: catalogue and index of French-language medical sites (CISMeF)]

The Catalogue and index of French-language medical sites (CISMeF) is a medical portal that provides users with results as pertinent as possible according to their requirements, expectations, and context of use. Indexing and single-term research are based on theMedical subject headings(MeSH) thesaurus. The integration of new medical terminology for indexing the catalogue's resources is intended to minimize false-negatives during searches and to contextualize the users' needs. The creation of a drug information portal makes more targeted research possible, with numerous entries according to user (physicians, pharmacists, chemists, and pharmacologists). For simplicity's sake, the catalogue's index of resources by different nomenclatures is not entirely displayed. The choice of display is left to the user, with MeSH only as the default. These multi-nomenclature tools should be applicable as well to electronic patient records. In this case, the objective is to improve patient care by better searches and identification of the information required during consultations and hospitalization.

French Infobutton: an academic and business perspective

Infobuttons are context-specific links between clinical information systems and other online information resources. The objective of this study is to describe a French Infobutton, which will be sold in the French-speaking Health Information market.

Cataloguing and displaying Web feeds from French language health sites: a Web 2.0 add-on to a health gateway

Among the numerous new functionalities of the Internet, commonly called Web 2.0, Web syndication illustrates the trend for better and faster information sharing. Web feeds (a.k.a RSS feeds), which were used mostly on weblogs at first, are now also widely used in academic, scientific and institutional websites such as PubMed. As very few French language feeds were listed or catalogued in the Health field by the year of 2007, it was decided to implement them in the quality-controlled health gateway CISMeF ([French] acronym for Catalogue and Index of French Language Health Resources on the Internet). Furthermore, making full use of the nature of Web syndication, a Web feed aggregator was put online in to provide a dynamic news gateway called "CISMeF actualités" (http://www.chu-rouen.fr/actualites/). This article describes the process to retrieve and implement the Web feeds in the catalogue and how its terminology was adjusted to describe this new content. It also describes how the aggregator was put online and the features of this news gateway. CISMeF actualités was built accordingly to the editorial policy of CISMeF. Only a part of the Web feeds of the catalogue were included to display the most authoritative sources. Web feeds were also grouped by medical specialties and by countries using the prior indexing of websites with MeSH terms and the so-called metaterms. CISMeF actualités now displays 131 Web feeds across 40 different medical specialities, coming from 5 different countries. It is one example, among many, that static hypertext links can now easily and beneficially be completed, or replaced, by dynamic display of Web content using syndication feeds.

Use of super-concepts to customize electronic medical records data display

Patient medical record systems (MRS) merely offer static applications, in which mostly unstructured text is linked to coded data. In these applications the more common presentation is a time oriented one, which does not allow easily for data and information retrieval. Concept oriented views based on supper-concepts (metaterms) initially defined in CISMeF to optimize Web medical search, was implemented in our MRS as specialties views. This work shows that these terminological tools are able to facilitate information retrieval.

Doc’UMVF: Two search tools to provide quality-controlled teaching resources in French to students and teachers

The UMVF project is a federation of medical teaching resources covering 32 medical schools in France. Today, the indexing of these resources is carried out manually by the CISMeF team at the University Hospital of Rouen. This indexing is based on MeSH thesaurus. We use a subset of SCORM metadata standard. This choice was defined in collaboration with the French Medical Virtual University consortium (French acronym: UMVF). Currently, with the UMVF searching tool (called Doc'UMVF), medical students can reach more than 3300 resources useful in their curriculum. Doc'UMVF is developed in close collaboration between the medical informatics laboratories of Rennes and Rouen. In this paper we present two complementary searching tools based on different methods and which are integrated and used to improve both the relevance and the coverage rate of the answers. A specific searching module has been built to retrieve specific resources concerning the National Medical Exam ENC ("Examen National Classant") is also available. Nevertheless, due to lack of time, numerous resources are not yet indexed. Therefore we have decided to use also automatic indexing method (Nomindex). This approach will be improved by further research works, resulting from Rouen and Geneva teams. After having built a searching meta-motor, our objective is to develop a meta-tool intended to index the whole set of digital pedagogical resources produced by the UMVF framework. This manual re-indexing will be carried out only for the most important resources (national references), with a more or less fine granularity.

Seal of transparency heritage in the CISMeF quality-controlled health gateway

Background

It is an absolute necessity to continually assess the quality of health information on the Internet. Quality-controlled subject gateways are Internet services which apply a selected set of targeted measures to support systematic resource discovery.

Methods

The CISMeF health gateway became a contributor to the MedCIRCLE project to evaluate 270 health information providers. The transparency heritage consists of using the evaluation performed on providers that are referenced in the CISMeF catalogue for evaluating the documents they publish, thus passing on the transparency label from the publishers to their documents.

Results

Each site rated in CISMeF has a record in the CISMeF database that generates an RDF into HTML file. The search tool Doc'CISMeF displays information originating from every publisher evaluated with a specific MedCIRCLE button, which is linked to the MedCIRCLE central repository. Starting with 270 websites, this trust heritage has led to 6,480 evaluated resources in CISMeF (49.8% of the 13,012 resources included in CISMeF).

Conclusion

With the MedCIRCLE project and transparency heritage, CISMeF became an explicit third party.

Level of evidence as a future gold standard for the content quality of health resources on the internet

OBJECTIVE

An assessment of the quality of health information on the Internet is an absolute necessity. In this study 'sensitive' information was defined as information found in documents published on the Internet, which could be used in a medical decision. For sensitive information, the main criterion chosen for the quality of the information was an indication of the level of evidence. A survey was conducted using the CISMeF health catalogue to assess how often a score of the level of evidence is mentioned in the information accessible on the Internet in French-language health resources.

METHODS

Since 1999, members of the CISMeF team have systematically been searching for all documents containing 'sensitive' information and verifying whether the level of evidence was explicitly indicated as a score at least once in the document.

RESULTS

As of June 2001, 10,190 resources were included in CISMeF; including 2964 textual 'sensitive' resources (29.1%). Out of all these resources, only 4.7% (95% confidence interval: 4.0 - 5.5%) indicated the level of evidence. A statistically significant difference in the prevalence of indicating the level of evidence according to resource types (e.g., 18.1% for guidelines compared to 0.0% for teaching material), year of publication (almost three times greater in 1997-2001 compared with 1990-1996) and publishers was observed.

CONCLUSION

As the number of people accessing the growing amount of information on the Internet is increasing daily, publishers have an ethical obligation to inform their readers about the validity of 'sensitive' information their sites contain. However, the vast majority of the French language Internet resources that were surveyed do not mention a score of the level of evidence for their sensitive information.

MedCIRCLE: collaboration for Internet rating, certification, labelling and evaluation of health information on the World-Wide-Web

We describe MedCIRCLE, an EU-funded semantic web project to implement the first steps towards a global, collaborative rating and guidance system for health information proposed in the MedCERTAIN project. In MedCIRCLE, three European gateway sites for consumer health information will implement the metadata vocabulary HIDDEL (Health Information Disclosure, Description and Evaluation Language). HIDDEL allows portals and gateways to make the results of their evaluations accessible as XML/RDF. The three participating national portals are: AQUMED (Agency for Quality in Medicine) patienten-information, de, COMB (Official Medical College of Barcelona) and CISMeF, a quality-controlled health gateway developed at Rouen University Hospital. Other health subject gateways, accreditation, or rating services are invited to join the collaboration simply by implementing HIDDEL on their gateways. Widespread implementation HIDDEL will allow intelligent agents or client-side software to harvest statements and opinions about the trustworthiness of other websites, assisting users in selecting trustworthy websites. The MedCIRCLE project builds on, expands and continues work on rating health information on the Internet piloted within the MedCERTAIN project. While MedCERTAIN provided the core technologies and software for rating and "trustmarking" health information, MedCIRCLE is built around these technologies and involves a wider medical community to assess health information, demonstrating the power of collaborative and interoperable evaluations in a semantic web environment. MedCIRCLE is a project with the overall objective to develop and promote technologies able to guide consumers to trustworthy health information on the Internet, to establish a global web of trust for networked health information, and to empower consumers to positively select high quality health information on the web. Other aims include refinement and expansion of HIDDEL, to become a standard vocabulary and interchange format for self- and third-party ratings of health information.

VUMeF: extending the French involvement in the UMLS Metathesaurus

A considerable number of robust vocabularies and thesauri have been developed for the healthcare and biomedical domain. No single vocabulary, however; provides complete coverage of the information needs from a public health perspective. The results of an investigation of vocabulary sources for the development of a comprehensive controlled vocabulary for the public health domain at the Centers for Disease Control and Prevention (CDC) is presented.

Reading factor: a bibliometric tool to manage a virtual library

Among the many bibliometric criteria used to evaluate biomedical journals, the impact factor is the most commonly used. Despite its limitations, it quantifies the influence of a journal on secondary publications. It does not however evaluate the practical usefulness of primary documents. Usefulness is field-related and varies greatly among specialities. We introduce a new bibliographic criterion, the "reading factor", and define it as the ratio between the number of electronic consultations of a particular journal (i.e., number of clicks on a hyper-link) and the mean number of electronic consultations of all the journals studied (itself calculated by dividing the total number of electronic accesses by the number of journals in the database). We describe its observed distribution, relative to that of the impact factor, based on electronic consultation records from our University Hospital medical digital library, where full-text electronic versions of 45 major biomedical journals have been available since December 1997. From this analysis we found no correlation between the 1999 reading factor and the 1998 impact factor of these 45 journals, and we observed a dramatic change in the hierarchy of journals upon using the reading factor as the yardstick rather than the impact factor. Moreover, we describe how using the reading factor has helped in managing the collection of our University Hospital's virtual library. The selection of journals to be discarded from the virtual library for the year 2001 was based on journals' RF values and this process will repeated over the coming years. The reading factor also permits a cost-analysis of a virtual library.

CONCLUSION

The measurement of the reading factor is highly automated, practical and efficient. It appears as a new tool for electronic collection management by librarians, well fitting with economical data.

Doc'CISMEF: a search tool based on "encapsulated" MeSH thesaurus

In the year 2000, the Internet became a major source of health information for the health professional and the Netizen. The objective of Doc'CISMeF (D'C) was to create a powerful generic search tool based on an structured information model which â encapsulates' the MeSH thesaurus to index and retrieve quality health resources on the Internet. To index resources, D'C uses four sections in its information model: 'meta-term', keyword, subheading, and resource type. Two search options are available: simple and advanced. The simple search requires the end-user to input a single term or expression. If this term belongs to the D'C information structure model, it will be exploded. If not, a full-text search is performed. In the advanced search, complex searches are possible combining Boolean operators with meta-terms, keywords, subheadings and resource types. D'C uses two standard tools for organising information: the MeSH thesaurus and the Dublin Core metadata format. Resources included in D'C are described according to the following elements: title, author or creator, subject and keywords, description, publisher, date, resource type, format, identifier, and language.

Webmastering in academic institutions: a new job opportunity

In cyberspace, the health webmaster could be regarded as a virtual editor-in-chief, in charge of content and design. In this circumstance, he/she must follow quality criteria when building any resource. At the Rouen University Hospital (RUH), we have chosen "Net Scoring" as an effective tool to aid the design of a quality Web site. "Net Scoring" contains a list of 49 criteria which fall into eight categories: credibility, content, links, design, interactivity, quantitative aspects, ethics, and accessibility. The webmaster is the key element of the editorial board process. He/she must regularly monitor the Web site in order to retrieve information about whether the site is used and by whom: the method most commonly used is log analysis. At the RUH, an average of 9,000 unique machines visit our Web site each working day. Webmaster is a new job opportunity in academic institutions, in particular for medical informaticians and medical librarians both whom are information science professionals.

An assessment of the visibility of MeSH-indexed medical web catalogs through search engines

Manually indexed Internet health catalogs such as CliniWeb or CISMeF provide resources for retrieving high-quality health information. Users of these quality-controlled subject gateways are most often referred to them by general search engines such as Google, AltaVista, etc. This raises several questions, among which the following: what is the relative visibility of medical Internet catalogs through search engines? This study addresses this issue by measuring and comparing the visibility of six major, MeSH-indexed health catalogs through four different search engines (AltaVista, Google, Lycos, Northern Light) in two languages (English and French). Over half a million queries were sent to the search engines; for most of these search engines, according to our measures at the time the queries were sent, the most visible catalog for English MeSH terms was CliniWeb and the most visible one for French MeSH terms was CISMeF.

Web impact factor: a bibliometric criterion applied to medical informatics societies' web sites

Several methods are available to evaluate and compare medical journals. The most popular is the journal Impact Factor, derived from averaging counts of citations to articles. Ingwersen adapted this method to assess the attractiveness of Web sites, defining the external Web Impact Factor (WIF) to be the number of external pages containing a link to a given Web site. This paper applies the WIF to 43 medical informatics societies' Web sites using advanced search engine queries to obtain the necessary link counts. The WIF was compared to the number of publications available in the Medline bibliographic database in medical informatics in these 43 countries. Between these two metrics, the observed Pearson correlation was 0.952 (p < 0.01) and the Spearman rank correlation was 0.548 (p < 0.01) showing in both cases a positive and strong significant correlation. The WIF of medicalm informatics society's Web site is statistically related to national productivity and discrepancies can be used to indicate countries where there are either weak medical informatics associations, or ones that do not make optimal use of the Web.

A preliminary study to assess a French code of ethics for health teaching resources on the Internet

BACKGROUND

Constant assessment of the quality of health information on the Internet is an absolute necessity as peer review is often lacking in this media.

OBJECTIVE

To develop a simple and easy French Code of Ethics, which will enable medical students to judge quality of health information in teaching material available on the Internet.

DESIGN

Three medical informaticians selected ten criteria from previously established codes of ethics from Europe and the USA. This instrument was tested on a sample of 30 health Internet teaching resources.

RESULTS

For the panel of experts, chance corrected inter-observer agreement (kappa) for quality rating ranged from k = -0.19 and k = 0.33, demonstrating poor agreement among the raters.

CONCLUSION

If negative results of this preliminary study are confirmed by further research, this finding may detrimentally affect projects in Europe to accredit or certify Internet health resources.

Cost effectiveness of a medical digital library

The rapid increase in the price of electronic journals has made the optimization of collection management an urgent task. As there is currently no standard procedure for the evaluation of this problem, we applied the Reading Factor (RF), an electronically computed indicator used for consultation of individual articles. The aim of our study was to assess the cost effective impact of modifications in our digital library (i.e. change of access from the Intranet to the Internet or change in editorial policy). The digital OVID library at Rouen University Hospital continues to be cost-effective in comparison with the interlibrary loan costs. Moreover, when electronic versions are offered alongside a limited amount of interlibrary loans, a reduction in library costs was observed.

A search tool based on 'encapsulated' MeSH thesaurus to retrieve quality health resources on the internet

In the year 2001, the Internet has become a major source of health information for the health professional and the Netizen. The objective of Doc' CISMeF (D'C) was to create a powerful generic search tool based on a structured information model which 'encapsulates' the MeSH thesaurus to index and retrieve quality health resources on the Internet. To index resources, D'C uses four sections in its information model: 'meta-term', keyword, subheading, and resource type. Two search options are available: simple and advanced. The simple search requires the end-user to input a single term or expression. If this term belongs to the D'C information structure model, it will be exploded. If not, a full-text search is performed. In the advanced search, complex searches are possible combining Boolean operators with meta-terms, keywords, subheadings and resource types. D'C uses two standard tools for organising information: the MeSH thesaurus and the Dublin Core metadata format. Resources included in D'C are described according to the following elements: title, author or creator, subject and keywords, description, publishers, date, resource type, format, identifier, and language.

The contribution of morphological knowledge to French MeSH mapping for information retrieval

MeSH-indexed Internet health directories must provide a mapping from natural language queries to MeSH terms so that both health professionals and the general public can query their contents. We describe here the design of lexical knowledge bases for mapping French expressions to MeSH terms, and the initial evaluation of their contribution to Doc'CISMeF, the search tool of a MeSH-indexed directory of French-language medical Internet resources. The observed trend is in favor of the use of morphological knowledge as a moderate (approximately 5%) but effective factor for improving query to term mapping capabilities.

A study comparing centralized CD-ROM and decentralized intranet access to MEDLINE

OBJECTIVE

The purpose of this study was to evaluate the efficacy of a decentralized intranet access in each medical department as opposed to centralized unique MEDLINE access in the medical library.

DESIGN

A two-phase questionnaire to evaluate MEDLINE use was given to junior and senior physicians at Rouen University Hospital (RUH). Phase I (August-October 1996) corresponded to a time period when centralized access was the only means of access available and phase II (August-October 1997) to a time period following the introduction of decentralized intranet access.

RESULTS

A total of 168 physicians filled out at least one phase of the questionnaire, among whom 123 (73%) filled out both phases. Use of MEDLINE significantly increased in 1997 (average of 10.2+/-1.1 searches in three months) versus 1996 (average of 4.9+/-0.7 searches in three months, P<0.0001). The aim of searches changed, becoming significantly more care oriented in phase II (P<0.0001). The number of searches performed by the physicians alone increased (P<0.0001) and searches performed by the librarian decreased (P<0.0001) in phase II. The method of searches also changed, as searches by author (P< 0.0001), by journal (P = 0.0042), and by free word (P = 0.0027) increased in phase II. Knowledge of the following concepts of MEDLINE significantly increased: explosion (P<0.0001), scope note (P<0.0001), Abridged Index Medicus (AIM) journals (P<0.0001), Medical Subject Headings (MeSH) qualifier (P<0.0001), and focus (P<0.0001).

CONCLUSION

A decentralized intranet access to MEDLINE increased the number of searches and knowledge of this bibliographic database. MEDLINE intranet access modified the purpose and the methods of searching.

CISMeF: cataloque and index of French speaking health resources

In 1999, the Internet has become a major source of health information. The objective of CISMeF is to catalogue and index the main French-speaking sites and documents concerning health. Currently, the number of resources already totalled over 6,100 with a mean of 75 new sites each week. CISMeF contains a thematic index, including medical specialities and an alphabetic index. CISMeF uses two standard tools for organising information: the MeSH (Medical Subject Heading) thesaurus from the Medline bibliographic database (National Library of Medicine) and the Dublin Core metadata format. A brief description of the site is systematically added. CISMeF respects the Net Scoring, criteria to assess the quality of health information on the Internet. The CISMeF project fulfils a valuable tool for the French-speaking health community: 2,500 machines visit the Web site each working day.

CISMeF: a structured health resource guide

In 1999, the Internet has become a major source of health information. The objective of CISMeF is to catalogue and index the main French-speaking health resources. In September 1999, the number of indexed resources totaled over 7,100 with a mean of 75 new sites per week. CISMeF uses two standard tools for organizing information: the Medline bibliographic database MeSH thesaurus and the Dublin Core metadata format. Resources included in CISMeF are described by the following: title, author or creator, subject and keywords, description, publisher, date, resource type, format, identifier, and language. To index resources, CISMeF uses five levels of hierarchy: "meta-term", category, keyword, subheading, and resource type. CISMeF contains a thematic index, including medical specialities and an alphabetic index. CISMeF respects the Net Scoring, criteria to assess the quality of health information on the Internet. The CISMeF project offers a valuable tool for the French-speaking health community: 2,500 computer users visit the Web site each working day.

[CISMeF: catalog and index of French-speaking medical sites]

The Internet has now become a major source of health information. The aim of CISMeF is to catalogue and index the main French-speaking sites and documents concerning health. This project was initiated by Rouen University Hospital. Its URL is http://www.chu-rouen.fr/cismef. CISMeF covers all areas of health care and medical sciences, and is indexed both alphabetically and according to subject. It was set up on a Sun workstation under the Sun UNIX operating system and is entirely based on static HTML. By May 1999, the number of sites and documents indexed was already over 6,500, with a mean of 75 new sites added each week. CISMeF is updated via a five-step process: resource collection, filtering, description, classification, and indexing. The Net Scoring criteria are used to assess the quality of health information on the Internet. These criteria concern eight categories: credibility, content, links, design, interactivity, quantitative aspects, ethics and accessibility. CISMeF uses two standard tools to organize information: the MeSH (medical subject heading) thesaurus from the Medline reference database (National Library of Medicine, USA) and the Dublin core metadata format. The sites and documents included in CISMeF are described using the following elements from the Dublin core project: title, author or creator, subject and keywords, description, publisher, date, resource type, format, identifier, and language.

SETH: an expert system for the management on acute drug poisoning in adults

The aim of SETH is to give end-users specific advice concerning treatment and monitoring of adult drug poisoning. SETH is developed with an off the shelf expert system shell (KBMS) and runs on a microcomputer. Technical choices were done according to this analysis, financial considerations and portability. Currently, the database contains 1000 French drugs from 75 different toxicological classes. The SETH expert system simulates the expert reasoning, taking into account for each toxicological class delay, signs and dose. Two phases of evaluation were performed. The experimental implementation of Seth began in April 1992 in our Poison Control Centre. Since then, 1100 cases inputted by residents were analysed by SETH. The extension of the knowledge base to child poisoning began in March 1993.

Computer-aided decision support in hepatology

The aim of this study was to describe and to evaluate the publications of the last 30 years devoted to computer-aided decision support in clinical hepatology. The search used Medlars and references of articles. Computer-aided decision support (CADS) was classified in two categories: statistical systems and knowledge-based systems. Two specific questionnaires were used for methodologic evaluation, one for statistical systems and one for knowledge-based systems. They were filled out independently by two observers. A total of 31 papers were selected among 55 identified between 1960 and 1991. The maximum possible for the two scores was 24. The methodologic quality ranged from 4 to 22 (median, 12) for statistical systems and from 8 to 12 (median, 9) for knowledge-based systems. The poor level of methodology could explain in part the lack of utilization of computer-aided decision support in the daily clinical practice of hepatologists.

[Computer-assisted model of weight gain during pregnancy]

The goal of this study was to modelize the evolution of "ideal" weight gain during pregnancy and to generate automatically the appropriate diet. This computerized model has been developed on a microcomputer and has two units: the first unit calculates the "ideal" weight gain during pregnancy, based on the curves of Rosso which show that weight gain is not linear with term and depends of the prepregnancy weight. The second unit calculates the appropriate diet which is depending for the first visit on height, prepregnancy weight and weight gain and for the followed visits on weight gain and the diet situation of the previous visit. The next step will be the medical evaluation of this computer-aided modelization of weight gain during pregnancy.

[Development of an expert system as a teaching aid in the serological diagnosis of acute viral hepatitis]

The serological diagnosis of acute viral hepatitis is complex for both practitioners and students. We have developed an expert system to teach the serological diagnosis of acute viral hepatitis in order to help non-specialists in deciding on the diagnosis. Three levels of increasing difficulty have been devised. The procedure consists of the following steps: choice of first-line examinations; discontinuation of the examinations in cases where the diagnosis is ascertained; choice of the most discriminant examination in cases where the diagnosis is doubtful. The value of this computer-assisted teaching must be confirmed by randomized studies involving a group of students trained by "classical" methods and a group of students trained by a computer-assisted method.

[Value of digestive endoscopic examination in acquired immunodeficiency syndrome (45 cases)]

45 patients with acquired immunodeficiency syndrome (AIDS) were studied with gastrointestinal endoscopy (42 upper endoscopies and 12 colonoscopies). 28 patients had skin or buccal Kaposi's sarcoma with or without opportunistic infections and 17 had opportunistic infections. 12 patients out of 45 (27 per cent) had 1 or several Kaposi's sarcoma macroscopic gastrointestinal localisations, 12 documented by upper endoscopy and 4 by colonoscopy. Endoscopic biopsies confirmed the diagnosis 5 times out of 16 (31 per cent). 16 patients (38 per cent) had candidosis oesophagitis, 1 had ulcerative antritis, 2 had a erythematosus duodenitis, 6 had diffuse inflammatory mucosal colonic changes. The specific abnormalities documented by histology were 1 case of total villous atrophy and 2 cases of cytomegalovirus colitis. The patients with cutaneous Kaposi's sarcoma have more often shown gastrointestinal Kaposi's sarcoma than the patients without Kaposi's sarcoma. The prevalence of tumoral or major (diffuse candidosis oesophagitis, ulcerative antritis, active colitis) endoscopic abnormalities documented by upper endoscopy was 40 per cent (10 cases out of 25) in asymptomatic patients, 43 per cent in diarrheic patients (3 cases out of 7) and 60 per cent (6 cases out of 10) in patients with dysphagia, epigastralgic pain or vomiting and that of documented by colonoscopy was 75 per cent (3 cases out of 4) in patients with mucus or bloody stools and 14 per cent (1 case out of 7) in diarrheic patients. The main result of the endoscopy on AIDS patients has therefore been the diagnosis of visceral localisations of Kaposi's sarcoma. The appearance of the secondary major lesions related to opportunistic infections is unusual, especially in asymptomatic patients. A systematic endoscopy on these patients remains a disputed question.