Guidance for reporting analyses of metadata on electronic health record use

INTRODUCTION

Research on how people interact with electronic health records (EHRs) increasingly involves the analysis of metadata on EHR use. These metadata can be recorded unobtrusively and capture EHR use at a scale unattainable through direct observation or self-reports. However, there is substantial variation in how metadata on EHR use are recorded, analyzed and described, limiting understanding, replication, and synthesis across studies.

RECOMMENDATIONS

In this perspective, we provide guidance to those working with EHR use metadata by describing 4 common types, how they are recorded, and how they can be aggregated into higher-level measures of EHR use. We also describe guidelines for reporting analyses of EHR use metadata-or measures of EHR use derived from them-to foster clarity, standardization, and reproducibility in this emerging and critical area of research.

Changes in Physician Electronic Health Record Use With the Expansion of Telemedicine

Importance

Understanding the drivers of electronic health record (EHR) burden, including EHR time and patient messaging, may directly inform strategies to address physician burnout. Given the COVID-19-induced expansion of telemedicine-now used for a substantial proportion of ambulatory encounters-its association with EHR burden should be evaluated.

Objective

To measure the association of the telemedicine expansion with time spent working in the EHR and with patient messaging among ambulatory physicians before and after the onset of the COVID-19 pandemic.

Design, Setting, and Participants

This longitudinal cohort study analyzed weekly EHR metadata of ambulatory physicians at UCSF Health, a large academic medical center. The same EHR measures were compared for 1 year before the COVID-19 pandemic (August 2018-September 2019) with the same period 1 year after its onset (August 2020-September 2021). Multivariable regression models evaluating the association between level of telemedicine use and EHR use were then assessed after the onset of the pandemic. The sample included all physician-weeks with at least 1 scheduled half-day clinic in the 11 largest ambulatory specialties at UCSF Health. Data analyses were performed from March 1, 2022, through July 1, 2023.

Exposures

Physicians' weekly modality mix of either entirely face-to-face visits, mixed modalities, or entirely telemedicine.

Main Outcomes and Measures

The EHR time during and outside of patient scheduled hours (PSHs), time spent documenting (normalized per 8 PSHs), and electronic messages sent to and received from patients.

Results

The study sample included 1052 physicians (437 [41.5%] men and 615 [58.5%] women) during 115 weeks, which provided 35 697 physician-week observations. Comparing the period before to the period after pandemic onset showed that physician time spent working in the EHR during PSHs increased from 4.53 to 5.46 hours per 8 PSH (difference, 0.93; 95% CI, 0.87-0.98; P < 0.001); outside of PSHs, increased from 4.29 to 5.34 hours (difference, 1.04; 95% CI, 0.95-1.14; P < 0.001); and time documenting during and outside of PSHs increased from 6.35 to 8.18 hours (difference, 1.83; 95% CI, 1.72-1.94; P < 0.001). Mean weekly messages received from patients increased from 16.76 to 30.33, and messages sent to patients increased from 13.82 to 29.83. In multivariable models, weeks with a mix of face-to-face and telemedicine (β, 0.43; 95% CI, 0.31-0.55; P < .001) visits or entirely telemedicine (β, 0.91; 95% CI, 0.74-1.09; P < .001) had more EHR time during PSHs than all face-to-face weeks, with similar results for EHR time outside of PSHs. There was no association between telemedicine use and messages received from patients, whereas mixed modalities (β, -0.90; 95% CI, -1.73 to -0.08; P = .03) and all telemedicine (β, -4.06; 95% CI, -5.19 to -2.93; P < .001) were associated with fewer messages sent to patients compared with entirely face-to-face weeks.

Conclusions and Relevance

The findings of this longitudinal cohort study suggest that telemedicine is associated with greater physician time spent working in the EHR, both during and outside of scheduled hours, mostly documenting visits and not messaging patients. Health systems may need to adjust productivity expectations for physicians and develop strategies to address EHR documentation burden for physicians.

Physician Electronic Health Record Use After Changes in US Centers for Medicare & Medicaid Services Documentation Requirements

This cohort study examines changes in physician electronic health record (EHR) documentation time before and after changes in Centers for Medicare &amp;amp; Medicaid evaluation and management requirements.

On-Call Junior Neurosurgery Residents Spend 9 hours of Their On-Call Shift Actively Using the Electronic Health Record

BACKGROUND

The electronic health record (EHR) is central to clinical workflow, yet few studies to date have explored EHR usage patterns among neurosurgery trainees.

OBJECTIVE

To describe the amount of EHR time spent by postgraduate year (PGY)-2 and PGY-3 neurosurgery residents during on-call days and the distribution of EHR activities in which they engage.

METHODS

This cohort study used the EHR audit logs, time-stamped records of user activities, to review EHR usage of PGY-2 and PGY-3 neurosurgery residents scheduled for 1 or more on-call days across 2 calendar years at the University of California San Francisco. We focused on the PGY-2 and PGY-3, which, in our training program, represent the primary participants in the in-house on-call pool.

RESULTS

Over 723 call days, 12 different residents took at least one on-call shift. The median (IQR) number of minutes that residents spent per on-call shift actively using the EHR was 536.8 (203.5), while interacting with an average (SD) of 68.1 (14.7) patient charts. There was no significant difference between Active EHR Time between residents as PGY-2 and PGY-3 on paired t -tests. Residents spent the most time on the following EHR activities: patient reports, notes, order management, patient list, and chart review.

CONCLUSION

Residents spent, on average, 9 hours of their on-call shift actively using the EHR, and there was no improved efficiency as residents gained experience. We noted several areas of administrative EHR burden, which could be reduced.

Team is brain: leveraging EHR audit log data for new insights into acute care processes

OBJECTIVE

To determine whether novel measures of contextual factors from multi-site electronic health record (EHR) audit log data can explain variation in clinical process outcomes.

MATERIALS AND METHODS

We selected one widely-used process outcome: emergency department (ED)-based team time to deliver tissue plasminogen activator (tPA) to patients with acute ischemic stroke (AIS). We evaluated Epic audit log data (that tracks EHR user-interactions) for 3052 AIS patients aged 18+ who received tPA after presenting to an ED at three Northern California health systems (Stanford Health Care, UCSF Health, and Kaiser Permanente Northern California). Our primary outcome was door-to-needle time (DNT) and we assessed bivariate and multivariate relationships with six audit log-derived measures of treatment team busyness and prior team experience.

RESULTS

Prior team experience was consistently associated with shorter DNT; teams with greater prior experience specifically on AIS cases had shorter DNT (minutes) across all sites: (Site 1: -94.73, 95% CI: -129.53 to 59.92; Site 2: -80.93, 95% CI: -130.43 to 31.43; Site 3: -42.95, 95% CI: -62.73 to 23.17). Teams with greater prior experience across all types of cases also had shorter DNT at two sites: (Site 1: -6.96, 95% CI: -14.56 to 0.65; Site 2: -19.16, 95% CI: -36.15 to 2.16; Site 3: -11.07, 95% CI: -17.39 to 4.74). Team busyness was not consistently associated with DNT across study sites.

CONCLUSIONS

EHR audit log data offers a novel, scalable approach to measure key contextual factors relevant to clinical process outcomes across multiple sites. Audit log-based measures of team experience were associated with better process outcomes for AIS care, suggesting opportunities to study underlying mechanisms and improve care through deliberate training, team-building, and scheduling to maximize team experience.

Weekend Versus Weekday Asthma-Related Emergency Department Utilization

OBJECTIVE

To assess variation in asthma-related emergency department (ED) use between weekends and weekdays.

METHODS

Cross-sectional administrative claims-based analysis using California 2016 Medicaid data and Vermont 2016 and Massachusetts 2015 all-payer claims databases. We defined ED use as the rate of asthma-related ED visits per 100 child-years. A weekend visit was a visit on Saturday or Sunday, based on date of ED visit claim. We used negative binomial regression and robust standard errors to assess variation between weekend and weekday rates, overall and by age group.

RESULTS

We evaluated data from 398,537 patients with asthma. The asthma-related ED visit rate was slightly lower on weekends (weekend: 18.7 [95% confidence interval (CI): 18.3-19.0], weekday: 19.6 [95% CI, 19.3-19.8], P < .001). When stratifying by age group, 3- to 5-year-olds had higher rates of asthma-related ED visits on weekends than weekdays (weekend: 33.7 [95% CI, 32.6-34.7], weekday: 29.8 [95% CI, 29.1-30.5], P < .001) and 12- to 17-year-olds had lower rates of ED visits on weekends than weekdays (weekend: 13.0 [95% CI: 12.5-13.4], weekday: 16.3 [95% CI: 15.9-16.7], P < .001). In the other age groups (6-11, 18-21 years) there were not statistically significant differences between weekend and weekday rates (P > .05).

CONCLUSIONS

In this multistate analysis of children with asthma, we found limited overall variation in pediatric asthma-related ED utilization on weekends versus weekdays. These findings suggest that increasing access options during the weekend may not necessarily decrease asthma-related ED use.

Development and Testing of an Emergency Department Quality Measure for Pediatric Suicidal Ideation and Self-Harm

OBJECTIVE

To develop and test a new quality measure assessing timeliness of follow-up mental health care for youth presenting to the emergency department (ED) with suicidal ideation or self-harm.

METHODS

Based on a conceptual framework, evidence review, and a modified Delphi process, we developed a quality measure assessing whether youth 5 to 17 years old evaluated for suicidal ideation or self-harm in the ED and discharged to home had a follow-up mental health care visit within 7 days. The measure was tested in 4 geographically dispersed states (California, Pennsylvania, South Carolina, Tennessee) using Medicaid administrative data. We examined measure feasibility of implementation, variation, reliability, and validity. To test validity, adjusted regression models examined associations between quality measure scores and subsequent all-cause and same-cause hospital readmissions/ED return visits.

RESULTS

Overall, there were 16,486 eligible ED visits between September 1, 2014 and July 31, 2016; 53.5% of eligible ED visits had an associated mental health care follow-up visit within 7 days. Measure scores varied by state, ranging from 26.3% to 66.5%, and by youth characteristics: visits by youth who were non-White, male, and living in an urban area were significantly less likely to be associated with a follow-up visit within 7 days. Better quality measure performance was not associated with decreased reutilization.

CONCLUSIONS

This new ED quality measure may be useful for monitoring and improving the quality of care for this vulnerable population; however, future work is needed to establish the measure's predictive validity using more prevalent outcomes such as recurrence of suicidal ideation or deliberate self-harm.

Follow-Up After Asthma Emergency Department Visits and Its Relationship With Subsequent Asthma-Related Utilization

OBJECTIVE

To assess the association between follow-up after an asthma-related emergency department (ED) visit and the likelihood of subsequent asthma-related ED utilization.

METHODS

Using data from California Medicaid (2014-2016), and Vermont (2014-2016) and Massachusetts (2013-2015) all-payer claims databases, we identified asthma-related ED visits for patients ages 3 to 21. Follow-up was defined as a visit within 14 days with a primary care provider or an asthma specialist.

OUTCOME

asthma-related ED revisit after the initial ED visit. Models included logistic regression to assess the relationship between 14-day follow-up and the outcome at 60 and 365 days, and mixed-effects negative binomial regression to assess the relationship between 14-day follow-up and repeated outcome events (# ED revisits/100 child-years). All models accounted for zip-code level clustering.

RESULTS

There were 90,267 ED visits, of which 22.6% had 14-day follow-up. Patients with follow-up were younger and more likely to have commercial insurance, complex chronic conditions, and evidence of prior asthma. 14-day follow-up was associated with decreased subsequent asthma-related ED revisits at 60 days (5.7% versus 6.4%, P < .001) and at 365 days (25.0% versus 28.3%, P < 0.001). Similarly, 14-day follow-up was associated with a decrease in the rate of repeated subsequent ED revisits (66.7 versus 77.3 revisits/100 child-years; P < 0.001).

CONCLUSIONS

We found a protective association between outpatient 14-day follow-up and asthma-related ED revisits. This may reflect improved asthma control as providers follow the NHLBI guideline stepwise approach. Our findings highlight an opportunity for improvement, with only 22.6% of those with asthma-related ED visits having 14-day follow-up.

Ask About What Matters: An Intervention to Improve Accessible Advance Care Planning Documentation

CONTEXT

Advance care planning (ACP) informs future medical decision-making, especially for patients with advanced age or serious illness. For clinicians to act on these preferences, or continue the ACP conversation as illness progresses, documentation of ACP discussions must be readily accessible within the electronic health record (EHR).

OBJECTIVES

Develop an intervention to improve accessible ACP documentation for hospitalized patients and assess its impact on viewing and documentation of ACP conversations within a specific EHR location.

METHODS

Adult patients age 75 or older or with serious illness discharged during a two-year period were included. The EHR's ACP Navigator was targeted as the intended location for documenting ACP-related activities. We implemented a hospital-wide, multipronged intervention that included increased ACP Navigator visibility and a process for workflow-congruent ACP documentation. Accessible ACP documentation was measured by documentation within the ACP Navigator and was analyzed by interrupted time-series analysis. ACP Navigator access was measured by user audit logs.

RESULTS

After the intervention, 6703 of 16,117 (41.6%) patient encounters had accessible ACP documentation, compared to 3689 of 13,143 (28.1%) preintervention (P < .001). In the intervention's first month, accessible ACP documentation increased 5.3% (P < .001, CI 2.9%-7.6%), followed by a 1.3% monthly increase relative to the preintervention period (P < .001, CI 1.0%-1.6%). ACP Navigator access for patients with ACP documentation increased in the intervention period (52.2% vs. 39.8%, P < .001).

CONCLUSION

An institution-wide intervention significantly increased accessible ACP documentation within a centralized location of the EHR. EHR usability changes improved rates of accessible ACP documentation and subsequent views of this documentation.

Context is Key: Using the Audit Log to Capture Contextual Factors Affecting Stroke Care Processes

High quality patient care through timely, precise and efficacious management depends not only on the clinical presentation of a patient, but the context of the care environment to which they present. Understanding and improving factors that affect streamlined workflow, such as provider or department busyness or experience, are essential to improving these care processes, but have been difficult to measure with traditional approaches and clinical data sources. In this exploratory data analysis, we aim to determine whether such contextual factors can be captured for important clinical processes by taking advantage of non-traditional data sources like EHR audit logs which passively track the electronic behavior of clinical teams. Our results illustrate the potential of defining multiple measures of contextual factors and their correlation with key care processes. We illustrate this using thrombolytic (tPA) treatment for ischemic stroke as an example process, but the measurement approaches can be generalized to multiple scenarios.

Changes in Drug List Prices and Amounts Paid by Patients and Insurers

IMPORTANCE

High out-of-pocket drug costs can cause patients to skip treatment and worsen outcomes, and high insurer drug payments could increase premiums. Drug wholesale list prices have doubled in recent years. However, because of manufacturer discounts and rebates, the extent to which increases in wholesale list prices are associated with amounts paid by patients and insurers is poorly characterized.

OBJECTIVE

To determine whether increases in wholesale list prices are associated with increases in amounts paid by patients and insurers for branded medications.

DESIGN, SETTING, AND PARTICIPANTS

Cross-sectional retrospective study analyzing pharmacy claims for patients younger than 65 years in the IBM MarketScan Commercial Database and pricing data from SSR Health, LLC, between January 1, 2010, and December 31, 2016. Pharmacy claims analyzed represent claims of employees and dependents participating in employer health benefit programs belonging to large employers. Rebate data were estimated from sales data from publicly traded companies. Analysis focused on the top 5 patent-protected specialty and 9 traditional brand-name medications with the highest total drug expenditures by commercial insurers nationwide in 2014. Data were analyzed from July 2017 to July 2020.

EXPOSURES

Calendar year.

MAIN OUTCOMES AND MEASURES

Changes in inflation-adjusted amounts paid by patients and insurers for branded medications.

RESULTS

In this analysis of 14.4 million pharmacy claims made by 1.8 million patients from 2010-2016, median drug wholesale list price increased by 129% (interquartile range [IQR], 78%-133%), while median insurance payments increased by 64% (IQR, 28%-120%) and out-of-pocket costs increased by 53% (IQR, 42%-82%). The mean percentage of wholesale list price accounted for by discounts increased from 17% in 2010 to 21% in 2016, and the mean percentage of wholesale list price accounted for by rebates increased from 22% in 2010 to 24% in 2016. For specialty medications, median patient out-of-pocket costs increased by 85% (IQR, 73%-88%) from 2010 to 2016 after adjustment for inflation and 42% (IQR, 25%-53%) for nonspecialty medications. During that same period, insurer payments increased by 116% for specialty medications (IQR, 100%-127%) and 28% for nonspecialty medications (IQR, 5%-34%).

CONCLUSIONS AND RELEVANCE

This study's findings suggest that drug list prices more than doubled over a 7-year study period. Despite rising manufacturer discounts and rebates, these price increases were associated with large increases in patient out-of-pocket costs and insurer payments.

Identifying tests related to breast cancer care in claims data

To develop a method for calculating rates of testing for breast cancer recurrence in patients who have already undergone initial treatment for breast cancer, we calculated rates in a cohort of Medicare breast cancer patients and an age-matched noncancer cohort. We first used only tests with claims including diagnosis codes indicating invasive breast cancer and then used all tests regardless of diagnosis code. For each method, we calculated testing rates in the breast cancer cohort above the background rate in the noncancer population. The two methods provided similar estimates of testing prevalence and frequency, with exception of prevalence of CT.

Geographic Variation in Postoperative Imaging for Low-Risk Breast Cancer

Background: The objective of this study was to examine the presence and magnitude of US geographic variation in use rates of both recommended and high-cost imaging in young patients with early-stage breast cancer during the 18 month period after surgical treatment of their primary tumor. Methods: Using the Truven Health MarketScan Commercial Database, a descriptive analysis was conducted of geographic variation in annual rates of dedicated breast imaging and high-cost body imaging of 36,045 women aged 18 to 64 years treated with surgery for invasive unilateral breast cancer between 2010 and 2012. Multivariate hierarchical analysis examined the relationship between likelihood of imaging and patient characteristics, with metropolitan statistical area (MSA) serving as a random effect. Patient characteristics included age group, BRCA1/2 carrier status, family history of breast cancer, combination of breast surgery type and radiation therapy, drug therapy, and payer type. All MSAs in the United States were included, with areas outside MSAs within a given state aggregated into a single area for analytic purposes. Results: Descriptive analysis of rates of imaging use and intensity within MSA regions revealed wide geographic variation, irrespective of treatment cohort or age group. Increased probability of recommended postoperative dedicated breast imaging was primarily associated with age and treatment including both surgery and radiation therapy, followed by MSA region (odds ratio, 1.42). Increased probability of PET use-a high-cost imaging modality for which postoperative routine use is not recommended in the absence of specific clinical findings-was primarily associated with surgery type followed by MSA region (odds ratio, 1.82). Conclusions: In patients with breast cancer treated for low-risk disease, geography has effects on the rates of posttreatment imaging, suggesting that some patients are not receiving beneficial dedicated breast imaging, and high-cost nonbreast imaging may not be targeted to those groups most likely to benefit.

Age- and Lesion-Related Comorbidity Burden Among US Adults With Congenital Heart Disease: A Population-Based Study

Background As patients with congenital heart disease (CHD) are living longer, understanding the comorbidities they develop as they age is increasingly important. However, there are no published population-based estimates of the comorbidity burden among the US adult patients with CHD. Methods and Results Using the IBM MarketScan commercial claims database from 2010 to 2016, we identified adults aged ≥18 years with CHD and 2 full years of continuous enrollment. These were frequency matched with adults without CHD within categories jointly defined by age, sex, and dates of enrollment in the database. A total of 40 127 patients with CHD met the inclusion criteria (mean [SD] age, 36.8 [14.6] years; and 48.2% were women). Adults with CHD were nearly twice as likely to have any comorbidity than those without CHD (P<0.001). After adjusting for covariates, patients with CHD had a higher prevalence risk ratio for "previously recognized to be common in CHD" (risk ratio, 9.41; 95% CI, 7.99-11.1), "other cardiovascular" (risk ratio, 1.73; 95% CI, 1.66-1.80), and "noncardiovascular" (risk ratio, 1.47; 95% CI, 1.41-1.52) comorbidities. After adjusting for covariates and considering interaction with age, patients with severe CHD had higher risks of previously recognized to be common in CHD and lower risks of other cardiovascular comorbidities than age-stratified patients with nonsevere CHD. For noncardiovascular comorbidities, the risk was higher among patients with severe than nonsevere CHD before, but not after, the age of 40 years. Conclusions Our data underscore the unique clinical needs of adults with CHD compared with their peers. Clinicians caring for CHD may want to use a multidisciplinary approach, including building close collaborations with internists and specialists, to help provide appropriate care for the highly prevalent noncardiovascular comorbidities.

Depression, Anxiety, and Emergency Department Use for Asthma

BACKGROUND AND OBJECTIVES

Asthma is responsible for ∼1.7 million emergency department (ED) visits annually in the United States. Studies in adults have shown that anxiety and depression are associated with increased asthma-related ED use. Our objective was to assess this association in pediatric patients with asthma.

METHODS

We identified patients aged 6 to 21 years with asthma in the Massachusetts All-Payer Claims Database for 2014 to 2015 using International Classification of Diseases, Ninth and 10th Revision codes. We examined the association between the presence of anxiety, depression, or comorbid anxiety and depression and the rate of asthma-related ED visits per 100 child-years using bivariate and multivariable analyses with negative binomial regression.

RESULTS

Of 65 342 patients with asthma, 24.7% had a diagnosis of anxiety, depression, or both (11.2% anxiety only, 5.8% depression only, and 7.7% both). The overall rate of asthma-related ED use was 17.1 ED visits per 100 child-years (95% confidence interval [CI]: 16.7-17.5). Controlling for age, sex, insurance type, and other chronic illness, patients with anxiety had a rate of 18.9 (95% CI: 17.0-20.8) ED visits per 100 child-years, patients with depression had a rate of 21.7 (95% CI: 18.3-25.0), and patients with both depression and anxiety had a rate of 27.6 (95% CI: 24.8-30.3). These rates were higher than those of patients who had no diagnosis of anxiety or depression (15.5 visits per 100 child-years; 95% CI: 14.5-16.4; P < .001).

CONCLUSIONS

Children with asthma and anxiety or depression alone, or comorbid anxiety and depression, have higher rates of asthma-related ED use compared with those without either diagnosis.

Validation of Prediction Models for Critical Care Outcomes Using Natural Language Processing of Electronic Health Record Data

Importance

Accurate prediction of outcomes among patients in intensive care units (ICUs) is important for clinical research and monitoring care quality. Most existing prediction models do not take full advantage of the electronic health record, using only the single worst value of laboratory tests and vital signs and largely ignoring information present in free-text notes. Whether capturing more of the available data and applying machine learning and natural language processing (NLP) can improve and automate the prediction of outcomes among patients in the ICU remains unknown.

Objectives

To evaluate the change in power for a mortality prediction model among patients in the ICU achieved by incorporating measures of clinical trajectory together with NLP of clinical text and to assess the generalizability of this approach.

Design, Setting, and Participants

This retrospective cohort study included 101 196 patients with a first-time admission to the ICU and a length of stay of at least 4 hours. Twenty ICUs at 2 academic medical centers (University of California, San Francisco [UCSF], and Beth Israel Deaconess Medical Center [BIDMC], Boston, Massachusetts) and 1 community hospital (Mills-Peninsula Medical Center [MPMC], Burlingame, California) contributed data from January 1, 2001, through June 1, 2017. Data were analyzed from July 1, 2017, through August 1, 2018.

Main Outcomes and Measures

In-hospital mortality and model discrimination as assessed by the area under the receiver operating characteristic curve (AUC) and model calibration as assessed by the modified Hosmer-Lemeshow statistic.

Results

Among 101 196 patients included in the analysis, 51.3% (n = 51 899) were male, with a mean (SD) age of 61.3 (17.1) years; their in-hospital mortality rate was 10.4% (n = 10 505). A baseline model using only the highest and lowest observed values for each laboratory test result or vital sign achieved a cross-validated AUC of 0.831 (95% CI, 0.830-0.832). In contrast, that model augmented with measures of clinical trajectory achieved an AUC of 0.899 (95% CI, 0.896-0.902; P < .001 for AUC difference). Further augmenting this model with NLP-derived terms associated with mortality further increased the AUC to 0.922 (95% CI, 0.916-0.924; P < .001). These NLP-derived terms were associated with improved model performance even when applied across sites (AUC difference for UCSF: 0.077 to 0.021; AUC difference for MPMC: 0.071 to 0.051; AUC difference for BIDMC: 0.035 to 0.043; P < .001) when augmenting with NLP at each site.

Conclusions and Relevance

Intensive care unit mortality prediction models incorporating measures of clinical trajectory and NLP-derived terms yielded excellent predictive performance and generalized well in this sample of hospitals. The role of these automated algorithms, particularly those using unstructured data from notes and other sources, in clinical research and quality improvement seems to merit additional investigation.