From cognition to the system: developing a multilevel taxonomy of patient safety in general practice

Patient safety classifications, taxonomies and ontologies, part 2: A systematic review on content coverage

BACKGROUND

Content coverage of patient safety ontology and classification systems should be evaluated to provide a guide for users to select appropriate ones for specific applications. In this review, we identified and compare content coverage of patient safety classifications and ontologies.

METHODS

We searched different databases and ontology/classification repositories to identify these classifications and ontologies. We included patient safety-related taxonomies, ontologies, classifications, and terminologies. We identified and extracted different concepts covered by these systems and mapped these concepts to international classification for patient safety (ICPS) and finally compared the content of these systems.

RESULTS

Finally, 89 papers (77 classifications or ontologies) were analyzed. Thirteen classifications have been developed to cover all medical domains. Among specific domain systems, most systems cover medication (16), surgery (8), medical devices (3), general practice (3), and primary care (3). The most common patient safety-related concepts covered in these systems include incident types (41), contributing factors/hazards (31), patient outcomes (29), degree of harm (25), and action (18). However, stage/phase (6), incident characteristics (5), detection (5), people involved (5), organizational outcomes (4), error type (4), and care setting (3) are some of the less covered concepts in these classifications/ontologies.

CONCLUSION

Among general systems, ICPS, World Health Organization's Adverse Reaction Terminology (WHO-ART), and Ontology of Adverse Events (OAE) cover most patient safety concepts and can be used as a gold standard for all medical domains. As a result, reporting systems could make use of these broad classifications, but the majority of their covered concepts are related to patient outcomes, with the exception of ICPS, which covers other patient safety concepts. However, the ICPS does not cover specialized domain concepts. For specific medical domains, MedDRA, NCC MERP, OPAE, ADRO, PPST, OCCME, TRTE, TSAHI, and PSIC-PC provide the broadest coverage of concepts. Many of the patient safety classifications and ontologies are not formally registered or available as formal classification/ontology in ontology repositories such as BioPortal. This study may be used as a guide for choosing appropriate classifications for various applications or expanding less developed patient safety classifications/ontologies. Furthermore, the same concepts are not represented by the same terms; therefore, the current study could be used to guide a harmonization process for existing or future patient safety classifications/ontologies.

Influences of early diagnostic suggestions on clinical reasoning

Previous research has highlighted the importance of physicians' early hypotheses for their subsequent diagnostic decisions. It has also been shown that diagnostic accuracy improves when physicians are presented with a list of diagnostic suggestions to consider at the start of the clinical encounter. The psychological mechanisms underlying this improvement in accuracy are hypothesised. It is possible that the provision of diagnostic suggestions disrupts physicians' intuitive thinking and reduces their certainty in their initial diagnostic hypotheses. This may encourage them to seek more information before reaching a diagnostic conclusion, evaluate this information more objectively, and be more open to changing their initial hypotheses. Three online experiments explored the effects of early diagnostic suggestions, provided by a hypothetical decision aid, on different aspects of the diagnostic reasoning process. Family physicians assessed up to two patient scenarios with and without suggestions. We measured effects on certainty about the initial diagnosis, information search and evaluation, and frequency of diagnostic changes. We did not find a clear and consistent effect of suggestions and detected mainly non-significant trends, some in the expected direction. We also detected a potential biasing effect: when the most likely diagnosis was included in the list of suggestions (vs. not included), physicians who gave that diagnosis initially, tended to request less information, evaluate it as more supportive of their diagnosis, become more certain about it, and change it less frequently when encountering new but ambiguous information; in other words, they seemed to validate rather than question their initial hypothesis. We conclude that further research using different methodologies and more realistic experimental situations is required to uncover both the beneficial and biasing effects of early diagnostic suggestions.

Patient safety classification, taxonomy and ontology systems: A systematic review on development and evaluation methodologies

INTRODUCTION

Patient safety classifications/ontologies enable patient safety information systems to receive and analyze patient safety data to improve patient safety. Patient safety classifications/ontologies have been developed and evaluated using a variety of methods. The purpose of this review was to discuss and analyze the methodologies for developing and evaluating patient safety classifications/ontologies.

METHODS

Studies that developed or evaluated patient safety classifications, terminologies, taxonomies, or ontologies were searched through Google Scholar, Google search engines, National Center for Biomedical Ontology (NCBO) BioPortal, Open Biological and Biomedical Ontology (OBO) Foundry and World Health Organization (WHO) websites and Scopus, Web of Science, PubMed, and Science Direct. We updated our search on 30 February 2021 and included all studies published until the end of 2020. Studies that developed or evaluated classifications only for patient safety and provided information on how they were developed or evaluated were included. Systems with covered patient safety terms (such as ICD-10) but are not specifically developed for patient safety were excluded. The quality and the risk of bias of studies were not assessed because all methodologies and criteria were intended to be covered. In addition, we analyzed the data through descriptive narrative synthesis and compared and classified the development and evaluation methods and evaluation criteria according to available development and evaluation approaches for biomedical ontologies.

RESULTS

We identified 84 articles that met all of the inclusion criteria, resulting in 70 classifications/ontologies, nine of which were for the general medical domain. The most papers were published in 2010 and 2011, with 8 and 7 papers, respectively. The United States (50) and Australia (23) have the most studies. The most commonly used methods for developing classifications/ontologies included the use of existing systems (for expanding or mapping) (44) and qualitative analysis of event reports (39). The most common evaluation methods were coding or classifying some safety report samples (25), quantitative analysis of incidents based on the developed classification (24), and consensus among physicians (16). The most commonly applied evaluation criteria were reliability (27), content and face validity (9), comprehensiveness (6), usability (5), linguistic clarity (5), and impact (4), respectively.

CONCLUSIONS

Because of the weaknesses and strengths of the development/evaluation methods, it is advised that more than one method for development or evaluation, as well as evaluation criteria, should be used. To organize the processes of developing classification/ontologies, well-established approaches such as Methontology are recommended. The most prevalent evaluation methods applied in this domain are well fitted to the biomedical ontology evaluation methods, but it is also advised to apply some evaluation approaches such as logic, rules, and Natural language processing (NLP) based in combination with other evaluation approaches. This research can assist domain researchers in developing or evaluating domain ontologies using more complete methodologies. There is also a lack of reporting consistency in the literature and same methods or criteria were reported with different terminologies.

Qualitative ergonomics/human factors research in health care: Current state and future directions

The objective of this systematic review was to understand the current state of Ergonomics/Human Factors (E/HF) qualitative research in health care and to draw implications for future efforts. This systematic review identified 98 qualitative research papers published between January 2005 and August 2015 in the seven journals endorsed by the International Ergonomics Association with an impact factor over 1.0. The majority of the studies were conducted in hospitals and outpatient clinics, were focused on the work of formal health care professionals, and were classified as cognitive or organizational ergonomics. Interviews, focus groups, and observations were the most prevalent forms of data collection. Triangulation and data archiving were the dominant approaches to ensuring rigor. Few studies employed a formal approach to qualitative inquiry. Significant opportunities remain to enhance the use of qualitative research to advance systems thinking within health care.

Predictors of diagnostic accuracy and safe management in difficult diagnostic problems in family medicine

OBJECTIVE

To investigate the role of information gathering and clinical experience on the diagnosis and management of difficult diagnostic problems in family medicine.

METHOD

Seven diagnostic scenarios including 1 to 4 predetermined features of difficulty were constructed and presented on a computer to 84 physicians: 21 residents in family medicine, 21 family physicians with 1 to 3 y in practice, and 42 family physicians with >or=10 y in practice. Following the Active Information Search process tracing approach, participants were initially presented with a patient description and presenting complaint and were subsequently able to request further information to diagnose and manage the patient. Evidence-based scoring criteria for information gathering, diagnosis, and management were derived from the literature and a separate study of expert opinion.

RESULTS

Rates of misdiagnosis were in accordance with the number of features of difficulty. Seventy-eight percent of incorrect diagnoses were followed by inappropriate management and 92% of correct diagnoses by appropriate management. Number of critical cues requested (cues diagnostic of any relevant differential diagnoses in a scenario) was a significant predictor of accuracy in 6 scenarios: 1 additional critical cue increased the odds of obtaining the correct diagnosis by between 1.3 (95% confidence interval [CI], 1.0-1.8) and 7.5 (95% CI, 3.2- 17.7), depending on the scenario. No effect of experience was detected on either diagnostic accuracy or management. Residents requested significantly more cues than experienced family physicians did.

CONCLUSIONS

Supporting the gathering of critical information has the potential to improve the diagnosis and management of difficult problems in family medicine.

Confidential reporting of patient safety events in primary care: results from a multilevel classification of cognitive and system factors

OBJECTIVE

To classify events of actual or potential harm to primary care patients using a multilevel taxonomy of cognitive and system factors.

METHODS

Observational study of patient safety events obtained via a confidential but not anonymous reporting system. Reports were followed up with interviews where necessary. Events were analysed for their causes and contributing factors using causal trees and were classified using the taxonomy. Five general medical practices in the West Midlands were selected to represent a range of sizes and types of patient population. All practice staff were invited to report patient safety events. Main outcome measures were frequencies of clinical types of events reported, cognitive types of error, types of detection and contributing factors; and relationship between types of error, practice size, patient consequences and detection.

RESULTS

78 reports were relevant to patient safety and analysable. They included 21 (27%) adverse events and 50 (64%) near misses. 16.7% (13/71) had serious patient consequences, including one death. 75.7% (59/78) had the potential for serious patient harm. Most reports referred to administrative errors (25.6%, 20/78). 60% (47/78) of the reports contained sufficient information to characterise cognition: "situation assessment and response selection" was involved in 45% (21/47) of these reports and was often linked to serious potential consequences. The most frequent contributing factor was work organisation, identified in 71 events. This included excessive task demands (47%, 37/71) and fragmentation (28%, 22/71).

CONCLUSIONS

Even though most reported events were near misses, events with serious patient consequences were also reported. Failures in situation assessment and response selection, a cognitive activity that occurs in both clinical and administrative tasks, was related to serious potential harm.