Developing an Audit and Feedback Dashboard for Family Physicians: User-Centered Design Process

BACKGROUND

Audit and feedback (A&F), the summary and provision of clinical performance data, is a common quality improvement strategy. Successful design and implementation of A&F-or any quality improvement strategy-should incorporate evidence-informed best practices as well as context-specific end user input.

OBJECTIVE

We used A&F theory and user-centered design to inform the development of a web-based primary care A&F dashboard. We describe the design process and how it influenced the design of the dashboard.

METHODS

Our design process included 3 phases: prototype development based on A&F theory and input from clinical improvement leaders; workshop with family physician quality improvement leaders to develop personas (ie, fictional users that represent an archetype character representative of our key users) and application of those personas to design decisions; and user-centered interviews with family physicians to learn about the physician's reactions to the revised dashboard.

RESULTS

The team applied A&F best practices to the dashboard prototype. Personas were used to identify target groups with challenges and behaviors as a tool for informed design decision-making. Our workshop produced 3 user personas, Dr Skeptic, Frazzled Physician, and Eager Implementer, representing common users based on the team's experience of A&F. Interviews were conducted to further validate findings from the persona workshop and found that (1) physicians were interested in how they compare with peers; however, if performance was above average, they were not motivated to improve even if gaps compared to other standards in their care remained; (2) burnout levels were high as physicians are trying to catch up on missed care during the pandemic and are therefore less motivated to act on the data; and (3) additional desired features included integration within the electronic medical record, and more up-to-date and accurate data.

CONCLUSIONS

We found that carefully incorporating data from user interviews helped operationalize generic best practices for A&F to achieve an acceptable dashboard that could meet the needs and goals of physicians. We demonstrate such a design process in this paper. A&F dashboards should address physicians' data skepticism, present data in a way that spurs action, and support physicians to have the time and capacity to engage in quality improvement work; the steps we followed may help those responsible for quality improvement strategy implementation achieve these aims.

Virtual Care and Emergency Department Use During the COVID-19 Pandemic Among Patients of Family Physicians in Ontario, Canada

Importance

The COVID-19 pandemic has played a role in increased use of virtual care in primary care. However, few studies have examined the association between virtual primary care visits and other health care use.

Objective

To evaluate the association between the percentage of virtual visits in primary care and the rate of emergency department (ED) visits.

Design, Setting, and Participants

This cross-sectional study used routinely collected administrative data and was conducted in Ontario, Canada. The sample comprised family physicians with at least 1 primary care visit claim between February 1 and October 31, 2021, and permanent Ontario residents who were alive as of March 31, 2021. All residents were assigned to physicians according to enrollment and billing data.

Exposure

Family physicians' virtual visit rate was the exposure. Physicians were stratified by the percentage of total visits that they delivered virtually (via telephone or video) during the study period (0% [100% in person], >0%-20%, >20%-40%, >40%-60%, >60%-80%, >80% to <100%, or 100%).

Main Outcomes and Measures

Population-level ED visit rate was calculated for each stratum of virtual care use. Multivariable regression models were used to understand the relative rate of patient ED use after adjusting for rurality of practice, patient characteristics, and 2019 ED visit rates.

Results

Data were analyzed for a total of 13 820 family physicians (7114 males [51.5%]; mean [SD] age, 50 [13.1] years) with 12 951 063 patients (6 714 150 females [51.8%]; mean [SD] age, 42.6 [22.9] years) who were attached to these physicians. Most physicians provided between 40% and 80% of care virtually. A higher percentage of the physicians who provided more than 80% of care virtually were 65 years or older, female individuals, and practiced in big cities. Patient comorbidity and morbidity were similar across strata of virtual care use. The mean (SD) number of ED visits was highest among patients whose physicians provided only in-person care (470.3 [1918.8] per 1000 patients) and was lowest among patients of physicians who provided more than 80% to less than 100% of care virtually (242.0 [800.3] per 1000 patients). After adjustment for patient characteristics, patients of physicians with more than 20% of visits delivered virtually had lower rates of ED visits compared with patients of physicians who provided more than 0% to 20% of care virtually (eg, >80% to <100% vs >0%-20% virtual visits in big cities: relative rate, 0.77%; 95% CI, 0.74%-0.81%). This pattern was unchanged across all rurality of practice strata and after adjustment for 2019 ED visit rates. In urban areas, there was a gradient whereby patients of physicians providing the highest level of virtual care had the lowest ED visit rates.

Conclusions and Relevance

Findings of this study show that patients of physicians who provided a higher percentage of virtual care did not have higher ED visit rates compared with patients of physicians who provided the lowest levels of virtual care. The findings refute the hypothesis that family physicians providing more care virtually during the pandemic resulted in higher ED use.

Protocol for a Case Control Study to Evaluate Oral Health as a Biomarker of Child Exposure to Adverse Psychosocial Experiences

Background: The early identification of children who have experienced adversity is critical for the timely delivery of interventions to improve coping and reduce negative consequences. Self-report is the usual practice for identifying children with exposure to adversity. However, physiological characteristics that signal the presence of disease or other exposures may provide a more objective identification strategy. This protocol describes a case–control study that assesses whether exposure to adversity is more common in children with tooth enamel anomalies compared to children without such anomalies. Methods: For 150 mother–child pairs from a pediatric dental clinic in Toronto, Canada, maternal interviews will assess the child’s adverse and resilience-building experiences. Per child, one (exfoliated or extracted) tooth will be assessed for suspected enamel anomalies. If anomalies are present, the child is a case, and if absent, the child is a control. Tooth assessment modalities will include usual practice for dental exams (visual assessment) and modalities with greater sensitivity to identify anomalies. Conclusion: If structural changes in children’s teeth are associated with exposure to adversity, routine dental exams could provide an opportunity to screen children for experiences of adversity. Affected children could be referred for follow-up.