Inpatient portal clusters: identifying user groups based on portal features

Lessons Learned From Developing Dashboards to Support Decision-Making for Community Opioid Response by Community Stakeholders: Mixed Methods and Multisite Study

BACKGROUND

Data dashboards are published tools that present visualizations; they are increasingly used to display data about behavioral health, social determinants of health, and chronic and infectious disease risks to inform or support public health endeavors. Dashboards can be an evidence-based approach used by communities to influence decision-making in health care for specific populations. Despite widespread use, evidence on how to best design and use dashboards in the public health realm is limited. There is also a notable dearth of studies that examine and document the complexity and heterogeneity of dashboards in community settings.

OBJECTIVE

Community stakeholders engaged in the community response to the opioid overdose crisis could benefit from the use of data dashboards for decision-making. As part of the Communities That HEAL (CTH) intervention, community data dashboards were created for stakeholders to support decision-making. We assessed stakeholders' perceptions of the usability and use of the CTH dashboards for decision-making.

METHODS

We conducted a mixed methods assessment between June and July 2021 on the use of CTH dashboards. We administered the System Usability Scale (SUS) and conducted semistructured group interviews with users in 33 communities across 4 states of the United States. The SUS comprises 10 five-point Likert-scale questions measuring usability, each scored from 0 to 4. The interview guides were informed by the technology adoption model (TAM) and focused on perceived usefulness, perceived ease of use, intention to use, and contextual factors.

RESULTS

Overall, 62 users of the CTH dashboards completed the SUS and interviews. SUS scores (grand mean 73, SD 4.6) indicated that CTH dashboards were within the acceptable range for usability. From the qualitative interview data, we inductively created subthemes within the 4 dimensions of the TAM to contextualize stakeholders' perceptions of the dashboard's usefulness and ease of use, their intention to use, and contextual factors. These data also highlighted gaps in knowledge, design, and use, which could help focus efforts to improve the use and comprehension of dashboards by stakeholders.

CONCLUSIONS

We present a set of prioritized gaps identified by our national group and list a set of lessons learned for improved data dashboard design and use for community stakeholders. Findings from our novel application of both the SUS and TAM provide insights and highlight important gaps and lessons learned to inform the design of data dashboards for use by decision-making community stakeholders.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04111939; https://clinicaltrials.gov/study/NCT04111939.

Effect of In-Person vs Video Training and Access to All Functions vs a Limited Subset of Functions on Portal Use Among Inpatients: A Randomized Clinical Trial

Importance

Inpatient portals provide patients with clinical data and information about their care and have the potential to influence patient engagement and experience. Although significant resources have been devoted to implementing these portals, evaluation of their effects has been limited.

Objective

To assess the effects of patient training and portal functionality on use of an inpatient portal and on patient satisfaction and involvement with care.

Design, Setting, and Participants

This randomized clinical trial was conducted from December 15, 2016, to August 31, 2019, at 6 noncancer hospitals that were part of a single health care system. Patients who were at least 18 years of age, identified English as their preferred language, were not involuntarily confined or detained, and agreed to be provided a tablet to access the inpatient portal during their stay were eligible for participation. Data were analyzed from May 1, 2019, to March 15, 2021.

Interventions

A 2 × 2 factorial intervention design was used to compare 2 levels of a training intervention (touch intervention, consisting of in-person training vs built-in video tutorial) and 2 levels of portal function availability (tech intervention) within an inpatient portal (all functions operational vs a limited subset of functions).

Main Outcomes and Measures

The primary outcomes were inpatient portal use, measured by frequency and comprehensiveness of use, and patients' satisfaction and involvement with their care.

Results

Of 2892 participants, 1641 were women (56.7%) with a median age of 47.0 (95% CI, 46.0-48.0) years. Most patients were White (2221 [76.8%]). The median Charlson Comorbidity Index was 1 (95% CI, 1-1) and the median length of stay was 6 (95% CI, 6-7) days. Notably, the in-person training intervention was found to significantly increase inpatient portal use (incidence rate ratio, 1.34 [95% CI, 1.25-1.44]) compared with the video tutorial. Patients who received in-person training had significantly higher odds of being comprehensive portal users than those who received the video tutorial (odds ratio, 20.75 [95% CI, 16.49-26.10]). Among patients who received the full-tech intervention, those who also received the in-person intervention used the portal more frequently (incidence rate ratio, 1.36 [95% CI, 1.25-1.48]) and more comprehensively (odds ratio, 22.52; [95% CI, 17.13-29.62]) than those who received the video tutorial. Patients who received in-person training had higher odds (OR, 2.01 [95% CI, 1.16-3.50]) of reporting being satisfied in the 6-month postdischarge survey. Similarly, patients who received the full-tech intervention had higher odds (OR, 2.06 [95%CI, 1.42-2.99]) of reporting being satisfied in the 6-month postdischarge survey.

Conclusions and Relevance

Providing in-person training or robust portal functionality increased inpatient engagement with the portal during the hospital stay. The effects of the training intervention suggest that providing personalized training to support use of this health information technology can be a powerful approach to increase patient engagement via portals.

Trial Registration

ClinicalTrials.gov Identifier: NCT02943109.

Outpatient portal use among pregnant individuals: Cross-sectional, temporal, and cluster analysis of use

Background

Outpatient portal technology can improve patient engagement. For pregnant individuals, the level of engagement could have important implications for maternal and infant outcomes.

Objective

This study: (1) cross-sectionally and temporally characterized the outpatient portal use among pregnant individuals seen at our academic medical center; and (2) identified clusters of the outpatient portal user groups based on the cross-sectional and temporal patterns of use.

Methods

We used outpatient portal server-side log files to execute a hierarchical clustering algorithm to group 7663 pregnant individuals based on proportions of outpatient portal function use. Post-hoc analyses were performed to further assess outpatient portal use on key encounter characteristics.

Results

The most frequently used functions were MyRecord (access personal health information), Visits (manage appointments), Messaging (send/receive messages), and Billing (view bills, insurance information). Median outpatient portal function use plateaued by the third trimester. Four distinct clusters were identified among all pregnant individuals: “Schedulers,” “Resulters,” “Intense Digital Engagers,” and “Average Users.” Post-hoc analyses revealed that the use of the Visits function increased and the use of the MyRecord function decreased over time among clusters.

Conclusions

Our identification of distinct cluster groups of outpatient portal users among pregnant individuals underscores the importance of avoiding the use of generalizations when describing how such patients might engage with patient-facing technologies such as an outpatient portal. These results can be used to improve user experience and training with outpatient portal functions and may educate maternal health providers on patient engagement with the outpatient portal.

Differential Effects of Outpatient Portal User Status on Inpatient Portal Use: Observational Study

Background

The decision to use patient portals can be influenced by multiple factors, including individuals’ perceptions of the tool, which are based on both their personal skills and experiences. Prior experience with one type of portal may make individuals more comfortable with using newer portal technologies. Experienced outpatient portal users in particular may have confidence in their ability to use inpatient portals that have similar functionality. In practice, the use of both outpatient and inpatient portal technologies can provide patients with continuity of access to their health information across care settings, but the influence of one type of portal use on the use of other portals has not been studied.

Objective

This study aims to understand how patients’ use of an inpatient portal is influenced by outpatient portal use.

Methods

This study included patients from an academic medical center who were provided access to an inpatient portal during their hospital stays between 2016 and 2018 (N=1571). We analyzed inpatient portal log files to investigate how inpatient portal use varied by using 3 categories of outpatient portal users: prior users, new users, and nonusers.

Results

Compared with prior users (695/1571, 44.24%) of an outpatient portal, new users (214/1571, 13.62%) had higher use of a select set of inpatient portal functions (messaging function: incidence rate ratio [IRR] 1.33, 95% CI 1.06-1.67; function that provides access to the outpatient portal through the inpatient portal: IRR 1.34, 95% CI 1.13-1.58). Nonusers (662/1571, 42.14%), compared with prior users, had lower overall inpatient portal use (all active functions: IRR 0.68, 95% CI 0.60-0.78) and lower use of specific functions, which included the function to review vitals and laboratory results (IRR 0.51, 95% CI 0.36-0.73) and the function to access the outpatient portal (IRR 0.53, 95% CI 0.45-0.62). In comparison with prior users, nonusers also had lower odds of being comprehensive users (defined as using 8 or more unique portal functions; odds ratio [OR] 0.57, 95% CI 0.45-0.73) or composite users (defined as comprehensive users who initiated a 75th or greater percentile of portal sessions) of the inpatient portal (OR 0.42, 95% CI 0.29-0.60).

Conclusions

Patients’ use of an inpatient portal during their hospital stay appeared to be influenced by a combination of factors, including prior outpatient portal use. For new users, hospitalization itself, a major event that can motivate behavioral changes, may have influenced portal use. In contrast, nonusers might have lower self-efficacy in their ability to use technology to manage their health, contributing to their lower portal use. Understanding the relationship between the use of outpatient and inpatient portals can help direct targeted implementation strategies that encourage individuals to use these tools to better manage their health across care settings.

Simulation-derived best practices for clustering clinical data

INTRODUCTION

Clustering analyses in clinical contexts hold promise to improve the understanding of patient phenotype and disease course in chronic and acute clinical medicine. However, work remains to ensure that solutions are rigorous, valid, and reproducible. In this paper, we evaluate best practices for dissimilarity matrix calculation and clustering on mixed-type, clinical data.

METHODS

We simulate clinical data to represent problems in clinical trials, cohort studies, and EHR data, including single-type datasets (binary, continuous, categorical) and 4 data mixtures. We test 5 single distance metrics (Jaccard, Hamming, Gower, Manhattan, Euclidean) and 3 mixed distance metrics (DAISY, Supersom, and Mercator) with 3 clustering algorithms (hierarchical (HC), k-medoids, self-organizing maps (SOM)). We quantitatively and visually validate by Adjusted Rand Index (ARI) and silhouette width (SW). We applied our best methods to two real-world data sets: (1) 21 features collected on 247 patients with chronic lymphocytic leukemia, and (2) 40 features collected on 6000 patients admitted to an intensive care unit.

RESULTS

HC outperformed k-medoids and SOM by ARI across data types. DAISY produced the highest mean ARI for mixed data types for all mixtures except unbalanced mixtures dominated by continuous data. Compared to other methods, DAISY with HC uncovered superior, separable clusters in both real-world data sets.

DISCUSSION

Selecting an appropriate mixed-type metric allows the investigator to obtain optimal separation of patient clusters and get maximum use of their data. Superior metrics for mixed-type data handle multiple data types using multiple, type-focused distances. Better subclassification of disease opens avenues for targeted treatments, precision medicine, clinical decision support, and improved patient outcomes.

An electronic health record (EHR) log analysis shows limited clinician engagement with unsolicited genetic test results

How clinicians utilize medically actionable genomic information, displayed in the electronic health record (EHR), in medical decision-making remains unknown. Participating sites of the Electronic Medical Records and Genomics (eMERGE) Network have invested resources into EHR integration efforts to enable the display of genetic testing data across heterogeneous EHR systems. To assess clinicians' engagement with unsolicited EHR-integrated genetic test results of eMERGE participants within a large tertiary care academic medical center, we analyzed automatically generated EHR access log data. We found that clinicians viewed only 1% of all the eMERGE genetic test results integrated in the EHR. Using a cluster analysis, we also identified different user traits associated with varying degrees of engagement with the EHR-integrated genomic data. These data contribute important empirical knowledge about clinicians limited and brief engagements with unsolicited EHR-integrated genetic test results of eMERGE participants. Appreciation for user-specific roles provide additional context for why certain users were more or less engaged with the unsolicited results. This study highlights opportunities to use EHR log data as a performance metric to more precisely inform ongoing EHR-integration efforts and decisions about the allocation of informatics resources in genomic research.

Unsupervised machine learning and prognostic factors of survival in chronic lymphocytic leukemia

OBJECTIVE

Unsupervised machine learning approaches hold promise for large-scale clinical data. However, the heterogeneity of clinical data raises new methodological challenges in feature selection, choosing a distance metric that captures biological meaning, and visualization. We hypothesized that clustering could discover prognostic groups from patients with chronic lymphocytic leukemia, a disease that provides biological validation through well-understood outcomes.

METHODS

To address this challenge, we applied k-medoids clustering with 10 distance metrics to 2 experiments ("A" and "B") with mixed clinical features collapsed to binary vectors and visualized with both multidimensional scaling and t-stochastic neighbor embedding. To assess prognostic utility, we performed survival analysis using a Cox proportional hazard model, log-rank test, and Kaplan-Meier curves.

RESULTS

In both experiments, survival analysis revealed a statistically significant association between clusters and survival outcomes (A: overall survival, P = .0164; B: time from diagnosis to treatment, P = .0039). Multidimensional scaling separated clusters along a gradient mirroring the order of overall survival. Longer survival was associated with mutated immunoglobulin heavy-chain variable region gene (IGHV) status, absent Zap 70 expression, female sex, and younger age.

CONCLUSIONS

This approach to mixed-type data handling and selection of distance metric captured well-understood, binary, prognostic markers in chronic lymphocytic leukemia (sex, IGHV mutation status, ZAP70 expression status) with high fidelity.

Care Team Perspectives About an Inpatient Portal: Benefits and Challenges of Patients' Portal Use During Hospitalization

While current research about inpatient portals has focused largely on the patient perspective, it is also critical to consider the care team point of view, as support from these individuals is essential to successful portal implementation and use. We held brief in-person interviews with 433 care team members across a six-hospital health system to explore opinions about patients' use of an inpatient portal as perceived by care team members. Using the Inpatient Portal Evaluation Framework, we characterized benefits and challenges of portal use that care team members reported affected patients, themselves, and the collaborative work of these care teams with their patients. Interviewees noted inpatient portals can improve patient care and experience and also indicated room for improvement in portal use for hospitalized patients. Further understanding of the care team perspective is critical to inform approaches to inpatient portal implementation that best benefit both patients and providers.

Metrics for Outpatient Portal Use Based on Log File Analysis: Algorithm Development

BACKGROUND

Web-based outpatient portals help patients engage in the management of their health by allowing them to access their medical information, schedule appointments, track their medications, and communicate with their physicians and care team members. Initial studies have shown that portal adoption positively affects health outcomes; however, early studies typically relied on survey data. Using data from health portal applications, we conducted systematic assessments of patients' use of an outpatient portal to examine how patients engage with the tool.

OBJECTIVE

This study aimed to document the functionality of an outpatient portal in the context of outpatient care by mining portal usage data and to provide insights into how patients use this tool.

METHODS

Using audit log files from the outpatient portal associated with the electronic health record system implemented at a large multihospital academic medical center, we investigated the behavioral traces of a study population of 2607 patients who used the portal between July 2015 and February 2019. Patient portal use was defined as having an active account and having accessed any portal function more than once during the study time frame.

RESULTS

Through our analysis of audit log file data of the number and type of user interactions, we developed a taxonomy of functions and actions and computed analytic metrics, including frequency and comprehensiveness of use. We additionally documented the computational steps required to diagnose artifactual data and arrive at valid usage metrics. Of the 2607 patients in our sample, 2511 were active users of the patients portal where the median number of sessions was 94 (IQR 207). Function use was comprehensive at the patient level, while each session was instead limited to the use of one specific function. Only 17.45% (78,787/451,762) of the sessions were linked to activities involving more than one portal function.

CONCLUSIONS

In discussing the full methodological choices made in our analysis, we hope to promote the replicability of our study at other institutions and contribute to the establishment of best practices that can facilitate the adoption of behavioral metrics that enable the measurement of patient engagement based on the outpatient portal use.

Exploring How Professionals Within Agile Health Care Informatics Perceive Visualizations of Log File Analyses: Observational Study Followed by a Focus Group Interview

BACKGROUND

An increasing number of software companies work according to the agile software development method, which is difficult to integrate with user-centered design (UCD) practices. Log file analysis may provide opportunities for integrating UCD practices in the agile process. However, research within health care information technology mostly has a theoretical approach and is often focused on the researcher's interpretation of log file analyses.

OBJECTIVE

We aimed to propose a systematic approach to log file analysis in this study and present this to developers to explore how they react and interpret this approach in the context of a real-world health care information system, in an attempt to answer the following question: How may log file analyses contribute to increasing the match between the health care system and its users, within the agile development method, according to agile team members?

METHODS

This study comprised 2 phases to answer the research question. In the first phase, log files were collected from a health care information system and subsequently analyzed (summarizing sequential patterns, heat mapping, and clustering). In the second phase, the results of these analyses are presented to agile professionals during a focus group interview. The interpretations of the agile professionals are analyzed by open axial coding.

RESULTS

Log file data of 17,924 user sessions and, in total, 176,678 activities were collected. We found that the Patient Timeline is mainly visited, with 23,707 (23,707/176,678; 13.42%) visits in total. The main unique user session occurred in 5.99% (1074/17,924) of all user sessions, and this comprised Insert Measurement Values for Patient and Patient Timeline, followed by the page Patient Settings and, finally, Patient Treatment Plan. In the heat map, we found that users often navigated to the pages Insert Measurement Values and Load Messages Collaborate. Finally, in the cluster analysis, we found 5 clusters, namely, the Information-seeking cluster, the Collaborative cluster, the Mixed cluster, the Administrative cluster, and the Patient-oriented cluster. We found that the interpretations of these results by agile professionals are related to stating hypotheses (n=34), comparing paths (n=31), benchmarking (n=22), and prioritizing (n=17).

CONCLUSIONS

We found that analyzing log files provides agile professionals valuable insights into users' behavior. Therefore, we argue that log file analyses should be used within agile development to inform professionals about users' behavior. In this way, further UCD research can be informed by these results, making the methods less labor intensive. Moreover, we argue that these translations to an approach for further UCD research will be carried out by UCD specialists, as they are able to infer which goals the user had when going through these paths when looking at the log data.

Facilitating Organizational Change to Accommodate an Inpatient Portal

BACKGROUND

Patient portals are becoming more commonly used in the hospital inpatient setting. While the potential benefits of inpatient portals are acknowledged, there is a need for research that examines the challenges of portal implementation and the development of best practice approaches for successful implementation.

OBJECTIVE

We conducted this study to improve our understanding of the impact of the implementation of an inpatient portal on care team members in the context of a large academic medical center. Our study focused on the perspectives of nursing care team members about the inpatient portal.

METHODS

We interviewed care team members (n = 437) in four phases throughout the 2 years following implementation of an inpatient portal to learn about their ongoing perspectives regarding the inpatient portal and its impact on the organization.

RESULTS

The perspectives of care team members demonstrated a change in acceptance of the inpatient portal over time in terms of buy-in, positive workflow changes, and acknowledged benefits of the portal for both care team members and patients. There were also changes over time in perspectives of the care team in regards to (1) challenges with new technology, (2) impact of the portal on workflow, and (3) buy-in. Six strategies were identified as important for implementation success: (1) convene a stakeholder group, (2) offer continual portal training, (3) encourage shared responsibility, (4) identify champions, (5) provide provisioning feedback, and (6) support patient use.

CONCLUSION

Inpatient portals are recognized as an important tool for both patients and care team members, but the implementation of such a technology can create challenges. Given the perspectives care team members had about the impact of the inpatient portal, our findings suggest implementation requires attention to organizational changes that are needed to accommodate the tool and the development of strategies that can address challenges associated with the portal.

Exploring the Digital Divide: Age and Race Disparities in Use of an Inpatient Portal

Background: Age and race disparities in the use of new technologies-the digital divide-may be limiting the potential of patient-facing health information technology to improve health and health care. Objective: To investigate whether disparities exist in the use of patient portals designed specifically for the inpatient environment. Methods: Patients admitted to the six hospitals affiliated with a large, Midwestern academic medical center from July 2017 to July 2018 were provided with access to a tablet equipped with an inpatient portal and recruited to participate in the study (n = 842). Demographic characteristics of study enrollees were obtained from patients' electronic health records and surveys given to patients during their hospital stay. Log files from the inpatient portal were used to create a global measure of use and calculate use rates for specific portal features. Results: We found both age and race disparities in use of the inpatient portal. Patients aged 60-69 (45.3% difference, p < 0.001) and those over age 70 (36.7% difference, p = 0.04) used the inpatient portal less than patients aged 18-29. In addition, African American patients used the portal less than White patients (40.4% difference, p = 0.004). Discussion: These findings suggest that the availability of the technology alone may be insufficient to overcome barriers to use and that additional intervention may be needed to close the digital divide. Conclusions: We identified lower use of the inpatient portal among African American and older patients, relative to White and younger patients, respectively.

Using Electronic Health Records Data to Evaluate the Impact of Information Technology on Improving Health Equity: Evidence from China

This study evaluates the impact of health information technology in accessing medical resources and identifies its role in improving health equity. We used 262, 771 records from the electronic medical records and outpatient appointment systems of three clinics for logistic regression to analyze the impact of information technology on patients' access to medical care. We interviewed a few health professionals to gauge their reactions and to validate and understand our quantitative results. The proportion of inpatients affected by information technology is low, accounting for only 16.7% (N = 43, 870). The difference between rural and urban groups is statistically significant, and rural households are more susceptible to information technology. In addition, distance has a significant positive effect. We demonstrate an inverted U-shaped relationship between severity of disease and the impact of information technology. Moreover, our interview results are consistent with our quantitative results. Quantitative and interview results suggest that health information technology plays a positive role in accessing medical care for patients with rural household and those in remote areas. Meanwhile, this effect is complex for patients with different severities of illnesses. Governments and managers should vigorously promote health information technology for healthcare delivery in the future and focus their attention on patients with serious diseases.