ConLBS: An Attack Investigation Approach Using Contrastive Learning with Behavior Sequence

Attack investigation is an important research field in forensics analysis. Many existing supervised attack investigation methods rely on well-labeled data for effective training. While the unsupervised approach based on BERT can mitigate the issues, the high degree of similarity between certain real-world attacks and normal behaviors makes it challenging to accurately identify disguised attacks. This paper proposes ConLBS, an attack investigation approach that combines the contrastive learning framework and multi-layer transformer network to realize the classification of behavior sequences. Specifically, ConLBS constructs behavior sequences describing behavior patterns from audit logs, and a novel lemmatization strategy is proposed to map the semantics to the attack pattern layer. Four different augmentation strategies are explored to enhance the differentiation between attack and normal behavior sequences. Moreover, ConLBS can perform unsupervised representation learning on unlabeled sequences, and can be trained either supervised or unsupervised depending on the availability of labeled data. The performance of ConLBS is evaluated in two public datasets. The results show that ConLBS can effectively identify attack behavior sequences in the cases of unlabeled data or less labeled data to realize attack investigation, and can achieve superior effectiveness compared to existing methods and models.

Using electronic health record audit log data for research: insights from early efforts

Electronic health record audit logs capture a time-sequenced record of clinician activities while using the system. Audit log data therefore facilitate unobtrusive measurement at scale of clinical work activities and workflow as well as derivative, behavioral proxies (eg, teamwork). Given its considerable research potential, studies leveraging these data have burgeoned. As the field has matured, the challenges of using the data to answer significant research questions have come into focus. In this Perspective, we draw on our research experiences and insights from the broader audit log literature to advance audit log research. Specifically, we make 2 complementary recommendations that would facilitate substantial progress toward audit log-based measures that are: (1) transparent and validated, (2) standardized to allow for multisite studies, (3) sensitive to meaningful variability, (4) broader in scope to capture key aspects of clinical work including teamwork and coordination, and (5) linked to patient and clinical outcomes.

Measuring Collaboration Through Concurrent Electronic Health Record Usage: Network Analysis Study

BACKGROUND

Collaboration is vital within health care institutions, and it allows for the effective use of collective health care worker (HCW) expertise. Human-computer interactions involving electronic health records (EHRs) have become pervasive and act as an avenue for quantifying these collaborations using statistical and network analysis methods.

OBJECTIVE

We aimed to measure HCW collaboration and its characteristics by analyzing concurrent EHR usage.

METHODS

By extracting concurrent EHR usage events from audit log data, we defined concurrent sessions. For each HCW, we established a metric called concurrent intensity, which was the proportion of EHR activities in concurrent sessions over all EHR activities. Statistical models were used to test the differences in the concurrent intensity between HCWs. For each patient visit, starting from admission to discharge, we measured concurrent EHR usage across all HCWs, which we called temporal patterns. Again, we applied statistical models to test the differences in temporal patterns of the admission, discharge, and intermediate days of hospital stay between weekdays and weekends. Network analysis was leveraged to measure collaborative relationships among HCWs. We surveyed experts to determine if they could distinguish collaborative relationships between high and low likelihood categories derived from concurrent EHR usage. Clustering was used to aggregate concurrent activities to describe concurrent sessions. We gathered 4 months of EHR audit log data from a large academic medical center's neonatal intensive care unit (NICU) to validate the effectiveness of our framework.

RESULTS

There was a significant difference (P<.001) in the concurrent intensity (proportion of concurrent activities: ranging from mean 0.07, 95% CI 0.06-0.08, to mean 0.36, 95% CI 0.18-0.54; proportion of time spent on concurrent activities: ranging from mean 0.32, 95% CI 0.20-0.44, to mean 0.76, 95% CI 0.51-1.00) between the top 13 HCW specialties who had the largest amount of time spent in EHRs. Temporal patterns between weekday and weekend periods were significantly different on admission (number of concurrent intervals per hour: 11.60 vs 0.54; P<.001) and discharge days (4.72 vs 1.54; P<.001), but not during intermediate days of hospital stay. Neonatal nurses, fellows, frontline providers, neonatologists, consultants, respiratory therapists, and ancillary and support staff had collaborative relationships. NICU professionals could distinguish high likelihood collaborative relationships from low ones at significant rates (3.54, 95% CI 3.31-4.37 vs 2.64, 95% CI 2.46-3.29; P<.001). We identified 50 clusters of concurrent activities. Over 87% of concurrent sessions could be described by a single cluster, with the remaining 13% of sessions comprising multiple clusters.

CONCLUSIONS

Leveraging concurrent EHR usage workflow through audit logs to analyze HCW collaboration may improve our understanding of collaborative patient care. HCW collaboration using EHRs could potentially influence the quality of patient care, discharge timeliness, and clinician workload, stress, or burnout.

Electronic Medical Record Audit Time Logs as a Measure of Preoperative Work Before Total Joint Arthroplasty

BACKGROUND

In order to achieve rapid recovery total joint arthroplasty, surgeons and their teams are spending more time in the weeks before surgery to prepare patients. This study aims to quantify total knee and hip prearthroplasty work using retrospective electronic medical record (EMR) activity audit log analysis.

METHODS

EMR activity in 100 elective knee and 100 elective hip arthroplasty cases was performed using audit logs. Each mouse click and action in the EMR was recorded. The time between mouse clicks was calculated and summed for each member of the clinical team. Descriptive statistics and independent samples t-tests were conducted to quantify and compare total preoperative work (POW) between groups defined by gender, procedure, age, insurance type, or health literacy (P < .05).

RESULTS

The mean number of days defined in the prearthroplasty time period was 69.1 days (standard deviation [SD] 42.8; range 8-191). The mean time spent in each patient's chart in the prearthroplasty period was 76.8 (SD 47.8) minutes. Surgeon's work in the medical record accounted for 7.9 (SD 7.9) minutes, registered nurses 46.7 minutes (SD 39.1), physician extenders 10.8 minutes (SD 16.9), and licensed practical nurses and patient care technicians 9.8 minutes (SD 13.0). A significant difference was observed when groups were dichotomized based on age <65 and insurance provider type.

CONCLUSION

A considerable amount of POW is required to prepare patients for surgery from the clinic date one decides to pursue total joint arthroplasty and the day prior to surgery. Retrospective electronic time stamps from the EMR should represent the minimum time required for surgical preparation.

Mining tasks and task characteristics from electronic health record audit logs with unsupervised machine learning

OBJECTIVE

The characteristics of clinician activities while interacting with electronic health record (EHR) systems can influence the time spent in EHRs and workload. This study aims to characterize EHR activities as tasks and define novel, data-driven metrics.

MATERIALS AND METHODS

We leveraged unsupervised learning approaches to learn tasks from sequences of events in EHR audit logs. We developed metrics characterizing the prevalence of unique events and event repetition and applied them to categorize tasks into 4 complexity profiles. Between these profiles, Mann-Whitney U tests were applied to measure the differences in performance time, event type, and clinician prevalence, or the number of unique clinicians who were observed performing these tasks. In addition, we apply process mining frameworks paired with clinical annotations to support the validity of a sample of our identified tasks. We apply our approaches to learn tasks performed by nurses in the Vanderbilt University Medical Center neonatal intensive care unit.

RESULTS

We examined EHR audit logs generated by 33 neonatal intensive care unit nurses resulting in 57 234 sessions and 81 tasks. Our results indicated significant differences in performance time for each observed task complexity profile. There were no significant differences in clinician prevalence or in the frequency of viewing and modifying event types between tasks of different complexities. We presented a sample of expert-reviewed, annotated task workflows supporting the interpretation of their clinical meaningfulness.

CONCLUSIONS

The use of the audit log provides an opportunity to assist hospitals in further investigating clinician activities to optimize EHR workflows.

Using electronic health record audit logs to study clinical activity: a systematic review of aims, measures, and methods

OBJECTIVE

To systematically review published literature and identify consistency and variation in the aims, measures, and methods of studies using electronic health record (EHR) audit logs to observe clinical activities.

MATERIALS AND METHODS

In July 2019, we searched PubMed for articles using EHR audit logs to study clinical activities. We coded and clustered the aims, measures, and methods of each article into recurring categories. We likewise extracted and summarized the methods used to validate measures derived from audit logs and limitations discussed of using audit logs for research.

RESULTS

Eighty-five articles met inclusion criteria. Study aims included examining EHR use, care team dynamics, and clinical workflows. Studies employed 6 key audit log measures: counts of actions captured by audit logs (eg, problem list viewed), counts of higher-level activities imputed by researchers (eg, chart review), activity durations, activity sequences, activity clusters, and EHR user networks. Methods used to preprocess audit logs varied, including how authors filtered extraneous actions, mapped actions to higher-level activities, and interpreted repeated actions or gaps in activity. Nineteen studies validated results (22%), but only 9 (11%) through direct observation, demonstrating varying levels of measure accuracy.

DISCUSSION

While originally designed to aid access control, EHR audit logs have been used to observe diverse clinical activities. However, most studies lack sufficient discussion of measure definition, calculation, and validation to support replication, comparison, and cross-study synthesis.

CONCLUSION

EHR audit logs have potential to scale observational research but the complexity of audit log measures necessitates greater methodological transparency and validated standards.

Electronic Health Record Use among Ophthalmology Residents while on Call

BACKGROUND

As electronic health record (EHR) use becomes more widespread, detailed records of how users interact with the EHR, known as EHR audit logs, are being used to characterize the clinical workflows of physicians including residents. After-hours EHR use is of particular interest given its known association with physician burnout. Several studies have analyzed EHR audit logs for residents in other fields, such as internal medicine, but none thus far in ophthalmology. Here, we focused specifically on EHR use during on-call shifts outside of normal clinic hours.

METHODS

In this retrospective study, we analyzed raw EHR audit log data from on-call shifts for 12 ophthalmology residents at a single institution over the course of a calendar year. Data were analyzed to characterize total time spent using the EHR, clinical volume, diagnoses of patients seen on call, and EHR tasks.

RESULTS

Across all call shifts, the median and interquartile range (IQR) of the time spent logged into the EHR per shift were 88 and 131 minutes, respectively. The median (IQR) unique patient charts accessed per shift was 7 (9) patients. When standardized to per-hour measures, weekday evening shifts were the busiest call shifts with regard to both EHR use time and clinical volume. Total EHR use time and clinical volume were greatest in the summer months (July to September). Chart review comprised a majority (63.4%) of ophthalmology residents' on-call EHR activities.

CONCLUSION

In summary, EHR audit logs demonstrate substantial call burden for ophthalmology residents outside of regular clinic hours. These data and future studies can be used to further characterize the clinical exposure and call burden of ophthalmology residents and could potentially have broader implications in the fields of physician burnout and education policy.