Identifying nurses' concern concepts about patient deterioration using a standard nursing terminology

Does nurse use of a standardized flowsheet to document communication with advanced providers provide a mechanism to detect pulse oximetry failures? A retrospective study of electronic health record data

BACKGROUND

Pulse oximetry guides clinical decisions, yet does not uniformly identify hypoxemia. We hypothesized that nursing documentation of notifying providers, facilitated by a standardized flowsheet for documenting communication to providers (physicians, nurse practitioners, and physician assistants), may increase when hypoxemia is present, but undetected by the pulse oximeter, in events termed "occult hypoxemia."

OBJECTIVE

To compare nurse documentation of provider notification in the 4 h preceding cases of occult hypoxemia, normal oxygenation, and evident hypoxemia confirmed by an arterial blood gas reading.

METHODS

We conducted a retrospective study using electronic health record data from patients with COVID-19 at five hospitals in a healthcare system with paired SpO2 and SaO2 readings (measurements within 10 min of oxygen saturation levels in arterial blood, SaO2, and by pulse oximetry, SpO2). We applied multivariate logistic regression to assess if having any nursing documentation of provider notification in the 4 h prior to a paired reading confirming occult hypoxemia was more likely compared to a paired reading confirming normal oxygen status, adjusting for characteristics significantly associated with nursing documentation. We applied conditional logistic regression to assess if having any nursing documentation of provider notification was more likely in the 4-hour window preceding a paired reading compared to the 4-hour window 24 h earlier separately for occult hypoxemia, visible hypoxemia, and normal oxygenation.

RESULTS

There were data from 1910 patients hospitalized with COVID-19 who had 44,972 paired readings and an average of 26.5 (34.5) nursing documentation of provider notification events. The mean age was 63.4 (16.2). Almost half (866/1910, 45.3 %) were White, 701 (36.7 %) were Black, and 239 (12.5 %) were Hispanic. Having any nursing documentation of provider notification was 46 % more common in the 4 h before an occult hypoxemia paired reading compared to a normal oxygen status paired reading (OR 1.46, 95 % CI: 1.28-1.67). Comparing the 4 h immediately before the reading to the 4 h one day preceding the paired reading, there was a higher likelihood of having any nursing documentation of provider notification for both evident (OR 1.45, 95 % CI 1.24-1.68) and occult paired readings (OR 1.26, 95 % CI 1.04-1.53).

CONCLUSION

This study finds that nursing documentation of provider notification significantly increases prior to confirmed occult hypoxemia, which has potential in proactively identifying occult hypoxemia and other clinical issues. There is potential value to encouraging standardized documentation of nurse concern, including communication to providers, to facilitate its inclusion in clinical decision-making.

Examining the Generalizability of Pretrained De-identification Transformer Models on Narrative Nursing Notes

BACKGROUND

 Narrative nursing notes are a valuable resource in informatics research with unique predictive signals about patient care. The open sharing of these data, however, is appropriately constrained by rigorous regulations set by the Health Insurance Portability and Accountability Act (HIPAA) for the protection of privacy. Several models have been developed and evaluated on the open-source i2b2 dataset. A focus on the generalizability of these models with respect to nursing notes remains understudied.

OBJECTIVES

 The study aims to understand the generalizability of pretrained transformer models and investigate the variability of personal protected health information (PHI) distribution patterns between discharge summaries and nursing notes with a goal to inform the future design for model evaluation schema.

METHODS

 Two pretrained transformer models (RoBERTa, ClinicalBERT) fine-tuned on i2b2 2014 discharge summaries were evaluated on our data inpatient nursing notes and compared with the baseline performance. Statistical testing was deployed to assess differences in PHI distribution across discharge summaries and nursing notes.

RESULTS

 RoBERTa achieved the optimal performance when tested on an external source of data, with an F1 score of 0.887 across PHI categories and 0.932 in the PHI binary task. Overall, discharge summaries contained a higher number of PHI instances and categories of PHI compared with inpatient nursing notes.

CONCLUSION

 The study investigated the applicability of two pretrained transformers on inpatient nursing notes and examined the distinctions between nursing notes and discharge summaries concerning the utilization of personal PHI. Discharge summaries presented a greater quantity of PHI instances and types when compared with narrative nursing notes, but narrative nursing notes exhibited more diversity in the types of PHI present, with some pertaining to patient's personal life. The insights obtained from the research help improve the design and selection of algorithms, as well as contribute to the development of suitable performance thresholds for PHI.

Diagnostic concordance among nursing clinical decision support system users: a pilot study

OBJECTIVE

To analyze the nursing diagnostic concordance among users of a clinical decision support system (CDSS), The Electronic Documentation System of the Nursing Process of the University of São Paulo (PROCEnf-USP®), structured according to the Nanda International, Nursing Intervention Classification and Nursing Outcome Classification (NNN) Taxonomy.

MATERIALS AND METHODS

This pilot, exploratory-descriptive study was conducted from September 2017 to January 2018. Participants were nurses, nurse residents, and nursing undergraduates. Two previously validated written clinical case studies provided participants with comprehensive initial assessment clinical data to be registered in PROCEnf-USP®. After having registered the clinical data in PROCEnf-USP®, participants could either select diagnostic hypotheses offered by the system or add diagnoses not suggested by the system. A list of nursing diagnoses documented by the participants was extracted from the system. The concordance was analyzed by Light's Kappa (K).

RESULTS

The research study included 37 participants, which were 14 nurses, 10 nurse residents, and 13 nursing undergraduates. Of the 43 documented nursing diagnoses, there was poor concordance (K = 0.224) for the diagnosis "Ineffective airway clearance" (00031), moderate (K = 0.591) for "Chronic pain" (00133), and elevated (K = 0.655) for "Risk for unstable blood glucose level" (00179). The other nursing diagnoses had poor or no concordance.

DISCUSSION

Clinical reasoning skills are essential for the meaningful use of the CDSS.

CONCLUSIONS

There was concordance for only 3 nursing diagnoses related to biological needs. The low level of concordance might be related to the clinical judgment skills of the participants, the written cases, and the sample size.

Diagnostic accuracy of clinical outcome prediction using nursing data in intensive care patients: A systematic review

BACKGROUND

Nursing data consist of observations of patients' conditions and information on nurses' clinical judgment based on critically ill patients' behavior and physiological signs. Nursing data in electronic health records were recently emphasized as important predictors of patients' deterioration but have not been systematically reviewed.

OBJECTIVE

We conducted a systematic review of prediction models using nursing data for clinical outcomes, such as prolonged hospital stay, readmission, and mortality in intensive care patients, compared to physiological data only. In addition, the type of nursing data used in prediction model developments was investigated.

DESIGN

A systematic review.

METHODS

PubMed, CINAHL, Cochrane CENTRAL, EMBASE, IEEE Xplore Digital Library, Web of Science, and Scopus were searched. Clinical outcome prediction models using nursing data for intensive care patients were included. Clinical outcomes were prolonged hospital stay, readmission, and mortality. Data were extracted from selected studies such as study design, data source, outcome definition, sample size, predictors, reference test, model development, model performance, and evaluation. The risk of bias and applicability was assessed using the Prediction model Risk of Bias Assessment Tool checklist. Descriptive summaries were produced based on paired forest plots and summary receiver operating characteristic curves.

RESULTS

Sixteen studies were included in the systematic review. The data types of predictors used in prediction models were categorized as physiological data, nursing data, and clinical notes. The types of nursing data consisted of nursing notes, assessments, documentation frequency, and flowsheet comments. The studies using physiological data as a reference test showed higher predictive performance in combined data or nursing data than in physiological data. The overall risk of bias indicated that most of the included studies have a high risk.

CONCLUSIONS

This study was conducted to identify and review the diagnostic accuracy of clinical outcome prediction using nursing data in intensive care patients. Most of the included studies developed models using nursing notes, and other studies used nursing assessments, documentation frequency, and flowsheet comments. Although the findings need careful interpretation due to the high risk of bias, the area under the curve scores of nursing data and combined data were higher than physiological data alone. It is necessary to establish a strategy in prediction modeling to utilize nursing data, clinical notes, and physiological data as predictors, considering the clinical context rather than physiological data alone.

REGISTRATION

The protocol for this study is registered with PROSPERO (registration number: CRD42021273319).

Detecting Language Associated With Home Healthcare Patient's Risk for Hospitalization and Emergency Department Visit

BACKGROUND

About one in five patients receiving home healthcare (HHC) services are hospitalized or visit an emergency department (ED) during a home care episode. Early identification of at-risk patients can prevent these negative outcomes. However, risk indicators, including language in clinical notes that indicate a concern about a patient, are often hidden in narrative documentation throughout their HHC episode.

OBJECTIVE

The aim of the study was to develop an automated natural language processing (NLP) algorithm to identify concerning language indicative of HHC patients' risk of hospitalizations or ED visits.

METHODS

This study used the Omaha System-a standardized nursing terminology that describes problems/signs/symptoms that can occur in the community setting. First, five HHC experts iteratively reviewed the Omaha System and identified concerning concepts indicative of HHC patients' risk of hospitalizations or ED visits. Next, we developed and tested an NLP algorithm to identify these concerning concepts in HHC clinical notes automatically. The resulting NLP algorithm was applied on a large subset of narrative notes (2.3 million notes) documented for 66,317 unique patients ( n = 87,966 HHC episodes) admitted to one large HHC agency in the Northeast United States between 2015 and 2017.

RESULTS

A total of 160 Omaha System signs/symptoms were identified as concerning concepts for hospitalizations or ED visits in HHC. These signs/symptoms belong to 31 of the 42 available Omaha System problems. Overall, the NLP algorithm showed good performance in identifying concerning concepts in clinical notes. More than 18% of clinical notes were detected as having at least one concerning concept, and more than 90% of HHC episodes included at least one Omaha System problem. The most frequently documented concerning concepts were pain, followed by issues related to neuromusculoskeletal function, circulation, mental health, and communicable/infectious conditions.

CONCLUSION

Our findings suggest that concerning problems or symptoms that could increase the risk of hospitalization or ED visit were frequently documented in narrative clinical notes. NLP can automatically extract information from narrative clinical notes to improve our understanding of care needs in HHC. Next steps are to evaluate which concerning concepts identified in clinical notes predict hospitalization or ED visit.

Mental Health Signs Relevant for an Assessment Tool Suitable for Student and Novice Nurses: A Document Analysis

Novice nurses' mental health assessment practice is characterised by lack of consistency, despite mental health assessment being a core issue in professional nursing and patient safety across health services. This study aimed to identify mental health signs relevant for an assessment tool suitable for student and novice nurses. A document analysis approach was applied, and content analysis was used to analyse data extracted from carefully selected documents. Four main categories of mental health issues were identified: risk issues, symptom issues, psychological issues and self-care issues. Mental health signs were thereafter grouped in ten sub-categories characterising mental health concerns. These were: risk concerns, psychotic concerns, mood, affect and energy concerns, substance use concerns, somatic concerns, perception concerns, communication concerns, cognitive concerns, anxiety concerns and self-care concerns. The identified signs are considered relevant for student and novice nurses to learn and can be further developed into a clinical assessment tool for use in nursing education to strengthen mental health assessment competence in nursing education.

Factors Influencing Clinician Trust in Predictive Clinical Decision Support Systems for In-Hospital Deterioration: Qualitative Descriptive Study

BACKGROUND

Clinician trust in machine learning-based clinical decision support systems (CDSSs) for predicting in-hospital deterioration (a type of predictive CDSS) is essential for adoption. Evidence shows that clinician trust in predictive CDSSs is influenced by perceived understandability and perceived accuracy.

OBJECTIVE

The aim of this study was to explore the phenomenon of clinician trust in predictive CDSSs for in-hospital deterioration by confirming and characterizing factors known to influence trust (understandability and accuracy), uncovering and describing other influencing factors, and comparing nurses' and prescribing providers' trust in predictive CDSSs.

METHODS

We followed a qualitative descriptive methodology conducting directed deductive and inductive content analysis of interview data. Directed deductive analyses were guided by the human-computer trust conceptual framework. Semistructured interviews were conducted with nurses and prescribing providers (physicians, physician assistants, or nurse practitioners) working with a predictive CDSS at 2 hospitals in Mass General Brigham.

RESULTS

A total of 17 clinicians were interviewed. Concepts from the human-computer trust conceptual framework-perceived understandability and perceived technical competence (ie, perceived accuracy)-were found to influence clinician trust in predictive CDSSs for in-hospital deterioration. The concordance between clinicians' impressions of patients' clinical status and system predictions influenced clinicians' perceptions of system accuracy. Understandability was influenced by system explanations, both global and local, as well as training. In total, 3 additional themes emerged from the inductive analysis. The first, perceived actionability, captured the variation in clinicians' desires for predictive CDSSs to recommend a discrete action. The second, evidence, described the importance of both macro- (scientific) and micro- (anecdotal) evidence for fostering trust. The final theme, equitability, described fairness in system predictions. The findings were largely similar between nurses and prescribing providers.

CONCLUSIONS

Although there is a perceived trade-off between machine learning-based CDSS accuracy and understandability, our findings confirm that both are important for fostering clinician trust in predictive CDSSs for in-hospital deterioration. We found that reliance on the predictive CDSS in the clinical workflow may influence clinicians' requirements for trust. Future research should explore the impact of reliance, the optimal explanation design for enhancing understandability, and the role of perceived actionability in driving trust.

Surgical nurses' responses to worry: A qualitative focus-group study in the Netherlands

BACKGROUND

Hospital nurses observe and respond to deterioration using the 'National Early Warning Score 2'. Surgical nurses are highly engaged in the early recognition of and response to deterioration. Responses to deterioration are based on deviating vital signs, while nurses also act on subjective indicators like worry. Scientific literature and (inter)national guidelines do not mention any information about acting upon worry.

OBJECTIVE

To gain an in-depth understanding of the actions nurses on surgical wards undertake to generate an appropriate response to nurses' worry when the 'National Early Warning Score 2' does not indicate deterioration.

METHOD

A qualitative focus-group study with surgical nurses working at a hospital in the Netherlands. Data was collected by focus-group interviews supported by vignettes and analysed thematically.

FINDINGS

Four focus-group interviews with a total of 20 participants were conducted between February and April 2020. Two sequential themes emerged: 'Searching for explanation and confirmation' and 'Responding by actively applying nursing interventions'. Nurses gathered additional information about the patient and searched for a reference point to place this information in perspective. Nurses also approached others for co-assessment and verification. However, nurses faced barriers in calling for medical assistance. They felt physicians did not take them seriously. After gathering additional information, nurses responded by applying nursing interventions to comfort the patient.

CONCLUSION

Nurses mainly try to formalise an in-depth understanding of their feeling of worry to convince a physician to accurately treat the patient. Spending much time on a search to this understanding leads to delays in escalating care.

Healthcare Process Modeling to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals): Development and evaluation of a conceptual framework

OBJECTIVE

There are signals of clinicians' expert and knowledge-driven behaviors within clinical information systems (CIS) that can be exploited to support clinical prediction. Describe development of the Healthcare Process Modeling Framework to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals).

MATERIALS AND METHODS

We employed an iterative framework development approach that combined data-driven modeling and simulation testing to define and refine a process for phenotyping clinician behaviors. Our framework was developed and evaluated based on the Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) predictive model to detect and leverage signals of clinician expertise for prediction of patient trajectories.

RESULTS

Seven themes-identified during development and simulation testing of the CONCERN model-informed framework development. The HPM-ExpertSignals conceptual framework includes a 3-step modeling technique: (1) identify patterns of clinical behaviors from user interaction with CIS; (2) interpret patterns as proxies of an individual's decisions, knowledge, and expertise; and (3) use patterns in predictive models for associations with outcomes. The CONCERN model differentiated at risk patients earlier than other early warning scores, lending confidence to the HPM-ExpertSignals framework.

DISCUSSION

The HPM-ExpertSignals framework moves beyond transactional data analytics to model clinical knowledge, decision making, and CIS interactions, which can support predictive modeling with a focus on the rapid and frequent patient surveillance cycle.

CONCLUSIONS

We propose this framework as an approach to embed clinicians' knowledge-driven behaviors in predictions and inferences to facilitate capture of healthcare processes that are activated independently, and sometimes well before, physiological changes are apparent.

Evaluation of the Accuracy of Nursing Diagnoses Determined by Users of a Clinical Decision Support System

PURPOSE

To analyze the accuracy of nursing diagnoses determined by users of a clinical decision support system (CDSS) and to identify the predictive factors of high/moderate diagnostic accuracy.

METHODS

This is an exploratory-descriptive study carried out from September 2017 to January 2018. Participants were nurses, resident nurses, and senior year undergraduates. Two written case studies provided the participants with the clinical data to fill out the assessment forms in the CDSS. The accuracy of the selected diagnostic labels was determined by a panel of experts using the Diagnostic Accuracy Scale, Version 2. Descriptive statistics were used to describe the level of accuracy according to each group of participants. Analysis of variance was used to compare the mean percentages of accuracy categories across groups. A linear regression model was used to identify the predictors of diagnostic accuracy. The significance level was 5%. The study was approved by the Ethics Committee.

RESULTS

Fifteen undergraduates, 10 residents, and 22 nurses were enrolled in the study. Although resident nurses and students had selected predominantly highly accurate diagnoses (51.8 ± 19.1 and 48.9 ± 27.4, respectively), and nurses had selected mostly diagnoses of moderate accuracy (54.7 ± 14.7), there were no differences in the accuracy level of selected diagnoses across groups. According to the linear regression model, each diagnosis added by the participants decreased the diagnostic accuracy by 2.09% (p = .030), and no experience or a low level of experience using the system decreased such diagnostic accuracy by 5.41% (p = .022).

CONCLUSIONS

The CDSS contributes to decision making about diagnoses of less experienced people. Adding diagnoses not indicated by the CDSS and experience with the system are predictors of diagnostic accuracy.

CLINICAL RELEVANCE

In-service education regarding the use of CDSSs seems to be crucial to improve users' clinical judgment and decision making.