Distance-Learning, ADHD Quality Improvement in Primary Care: A Cluster-Randomized Trial

Measuring quality-of-care in treatment of young children with attention-deficit/hyperactivity disorder using pre-trained language models

OBJECTIVE

To measure pediatrician adherence to evidence-based guidelines in the treatment of young children with attention-deficit/hyperactivity disorder (ADHD) in a diverse healthcare system using natural language processing (NLP) techniques.

MATERIALS AND METHODS

We extracted structured and free-text data from electronic health records (EHRs) of all office visits (2015-2019) of children aged 4-6 years in a community-based primary healthcare network in California, who had ≥1 visits with an ICD-10 diagnosis of ADHD. Two pediatricians annotated clinical notes of the first ADHD visit for 423 patients. Inter-annotator agreement (IAA) was assessed for the recommendation for the first-line behavioral treatment (F-measure = 0.89). Four pre-trained language models, including BioClinical Bidirectional Encoder Representations from Transformers (BioClinicalBERT), were used to identify behavioral treatment recommendations using a 70/30 train/test split. For temporal validation, we deployed BioClinicalBERT on 1,020 unannotated notes from other ADHD visits and well-care visits; all positively classified notes (n = 53) and 5% of negatively classified notes (n = 50) were manually reviewed.

RESULTS

Of 423 patients, 313 (74%) were male; 298 (70%) were privately insured; 138 (33%) were White; 61 (14%) were Hispanic. The BioClinicalBERT model trained on the first ADHD visits achieved F1 = 0.76, precision = 0.81, recall = 0.72, and AUC = 0.81 [0.72-0.89]. Temporal validation achieved F1 = 0.77, precision = 0.68, and recall = 0.88. Fairness analysis revealed low model performance in publicly insured patients (F1 = 0.53).

CONCLUSION

Deploying pre-trained language models on a variable set of clinical notes accurately captured pediatrician adherence to guidelines in the treatment of children with ADHD. Validating this approach in other patient populations is needed to achieve equitable measurement of quality of care at scale and improve clinical care for mental health conditions.

A Quality Improvement Initiative for Detection of Attention-Deficit/Hyperactivity Disorder in an Urban, Academic Safety Net Hospital

OBJECTIVE

Improve detection of Attention Deficit/Hyperactivity Disorder (ADHD) in a safety net, hospital-based, academic pediatric practice by optimizing screening with the Pediatric Symptom Checklist attention score (PSC-AS) and further evaluation with the Vanderbilt ADHD Diagnostic Rating Scale (VADRS).

METHODS

We implemented a multi-component intervention by (1) optimizing electronic medical record (EMR) features; (2) adjusting clinic operational workflow; and (3) creating a decision-making algorithm for pediatric primary care clinicians (PPCCs). We extracted 4 outcomes manually from the EMR (pediatrician acknowledgment of a positive PSC-AS, documentation of a plan for further evaluation, distribution of VADRS, and completion of at least 1 VADRS). Outcomes were measured monthly in run charts compared to the pre-intervention control period, and implementation was optimized with Plan-Do-Study-Act cycles.

RESULTS

PPCCs were significantly more likely to acknowledge a positive PSC-AS in the intervention versus control (65.3% vs 41.5%; p < 0.001), although this did not change documentation of a plan (70% vs 67.1%; p -value = 0.565). Significantly more children with a positive PSC-AS were distributed a parent or teacher VADRS in the intervention versus control (30.6% vs 17.7%; p -value = 0.0059), but the percentage of returned VADRS rating scales did not improve (12.9% vs 9.2%; p -value = 0.269).

CONCLUSION

Our ADHD detection quality improvement initiative improved use of the PSC-AS to identify attention problems and distribution of VADRS diagnostic rating scales, but additional interventions are needed to improve the completion of ADHD evaluations in primary care to ensure that children are appropriately identified and offered evidence-based care.

Assessment and Management of Attention-Deficit/Hyperactivity Disorder: Pediatric Resident Perspectives on Training and Practice

Attention-deficit/hyperactivity disorder (ADHD) is a common presenting concern in primary care. This study examined the relationship between pediatric residency training program characteristics and residents' ADHD knowledge, attitudes, and comfort in providing ADHD services. Given the familiarity that pediatric chief residents have with the training and experiences within their residency programs, a 30-item survey was mailed to pediatric chief residents. A total of 100 residents returned their surveys (response rate 49.5%) and were included in the descriptive quantitative and thematic qualitative analyses. The majority of participants rated their ADHD knowledge as at least average. However, approximately half of the participants were comfortable with screening, and less than half were comfortable with managing stimulant medication or behavioral treatments. Participants emphasized the importance of interprofessional collaboration, clinical experiences, and integrated ADHD education throughout training. These results emphasize the importance of improved training in screening, diagnosing, and managing ADHD to increase resident comfort regarding these practices.

Sustainability, spread, and scale in trials using audit and feedback: a theory-informed, secondary analysis of a systematic review

BACKGROUND

Audit and feedback (A&F) is a widely used implementation strategy to influence health professionals' behavior that is often tested in implementation trials. This study examines how A&F trials describe sustainability, spread, and scale.

METHODS

This is a theory-informed, descriptive, secondary analysis of an update of the Cochrane systematic review of A&F trials, including all trials published since 2011. Keyword searches related to sustainability, spread, and scale were conducted. Trials with at least one keyword, and those identified from a forward citation search, were extracted to examine how they described sustainability, spread, and scale. Results were qualitatively analyzed using the Integrated Sustainability Framework (ISF) and the Framework for Going to Full Scale (FGFS).

RESULTS

From the larger review, n = 161 studies met eligibility criteria. Seventy-eight percent (n = 126) of trials included at least one keyword on sustainability, and 49% (n = 62) of those studies (39% overall) frequently mentioned sustainability based on inclusion of relevant text in multiple sections of the paper. For spread/scale, 62% (n = 100) of trials included at least one relevant keyword and 51% (n = 51) of those studies (31% overall) frequently mentioned spread/scale. A total of n = 38 studies from the forward citation search were included in the qualitative analysis. Although many studies mentioned the need to consider sustainability, there was limited detail on how this was planned, implemented, or assessed. The most frequent sustainability period duration was 12 months. Qualitative results mapped to the ISF, but not all determinants were represented. Strong alignment was found with the FGFS for phases of scale-up and support systems (infrastructure), but not for adoption mechanisms. New spread/scale themes included (1) aligning affordability and scalability; (2) balancing fidelity and scalability; and (3) balancing effect size and scalability.

CONCLUSION

A&F trials should plan for sustainability, spread, and scale so that if the trial is effective, the benefits can continue. A deeper empirical understanding of the factors impacting A&F sustainability is needed. Scalability planning should go beyond cost and infrastructure to consider other adoption mechanisms, such as leadership, policy, and communication, that may support further scalability.

TRIAL REGISTRATION

Registered with Prospero in May 2022. CRD42022332606.

Perspectives of Rural Primary Care Clinicians on Pediatric Attention-Deficit/Hyperactivity Disorder Care

OBJECTIVE

Despite efficacious treatments, evidence-based guidelines, and increased availability of integrated behavioral health care, youth coping with attention-deficit/hyperactivity disorder (ADHD) receive suboptimal care. More research is needed to understand and address care gaps, particularly within rural health systems that face unique challenges. We conducted a qualitative study within a predominantly rural health system with a pediatric-integrated behavioral health care program to address research gaps and prepare for quality improvement initiatives, including primary care clinician (PCC) trainings and clinical decision support tools in the electronic health record (EHR).

METHOD

Semistructured interviews were conducted with 26 PCCs representing clinics within the health system. Interview guides were based on the Practical Robust Implementation and Sustainability Model to elicit PCC views regarding determinants of current practices and suggestions to guide quality improvement efforts. We used thematic analysis to identify patterns of responding that were common across participants.

RESULTS

PCCs identified several internal and external contextual factors as determinants of current practices. Of note, PCCs recommended increased access to continuing education trainings held in clinic over lunch and delivered in less than 30 minutes. Suggested improvements to the EHR included incorporating parent and teacher versions of the Vanderbilt Rating Scale into the EHR, documentation templates aligned with evidence-based guidelines, and alerts and suggestions to aid medication management during appointments.

CONCLUSION

Future research to identify implementation strategies to help rural PCCs adopt innovations are needed given the increased responsibility for managing ADHD care and intractable gaps in access to behavioral health care in rural regions.

Innovation in the pediatric electronic health record to realize a more effective platform

Commercial electronic health records (EHRs) were first developed to automate business processes. As EHRs developed, design principles focused on transferring existing paper-based documentation to comparable electronic forms. In addition, a strong industry focus on adult healthcare settings and quality measures has limited attention and resources for high priority EHR functionality needed for the unique health care of children. The objective of this paper is to provide a review of innovation in the EHR, that includes a variety of established and emerging technologies that may help realize a more effective EHR in child health settings. A more effective EHR would serve as an electronic hub. Existing EHR infrastructure could provide the foundation upon which new technologies and approaches branch and extend, enabling more rapid and customizable innovation to better meet shifting stakeholder and end-user needs. Among many areas for improvement, key goals of innovation could include technology that relieves ambulatory primary care clinician documentation burden, identifies needs, and supports improved care coordination and outcomes, focused on the following key areas: identification of child and family care needs, decision support, documentation, care coordination, and family communication.

Improving Care Management in Attention-Deficit/Hyperactivity Disorder: An RCT

OBJECTIVES

To compare the effectiveness of care management combined with a patient portal versus a portal alone for communication among children with attention-deficit/hyperactivity disorder (ADHD).

METHODS

Randomized controlled trial conducted at 11 primary care practices. Children aged 5 to 12 years old with ADHD were randomly assigned to care management + portal or portal alone. The portal included parent-reported treatment preferences and goals, medication side effects, and parent- and teacher-reported ADHD symptom scales. Care managers provided education to families; communicated quarterly with parents, teachers, and clinicians; and coordinated care. The main outcome, changes in the Vanderbilt Parent Rating Scale (VPRS) score as a measure of ADHD symptoms, was assessed using intention-to-treat analysis.

RESULTS

A total of 303 eligible children (69% male; 46% Black) were randomly assigned, and 273 (90%) completed the study. During the 9-month study, parents in the care management + portal arm communicated inconsistently with care managers (mean 2.2; range 0-6) but similarly used the portal (mean 2.3 vs 2.2) as parents in the portal alone arm. In multivariate models, VPRS scores decreased over time (Adjusted β = -.015; 95% confidence interval -0.023 to -0.07) in both groups, but there were no intervention-by-time effects (Adjusted β = .000; 95% confidence interval -0.011 to 0.012) between groups. Children who received ≥2 care management sessions had greater reductions in VPRS scores than those with fewer sessions.

CONCLUSIONS

Results did not provide evidence that care management combined with a patient portal was different from portal use alone among children with ADHD. Both groups demonstrated similar reductions in ADHD symptoms. Those families with greater care management engagement demonstrated greater reductions than those with less engagement.

Strategic Implementation Planning for Integrated Behavioral Health Services in Pediatric Primary Care

Background

Delivering physical and behavioral health services in a single setting is associated with improved quality of care and reduced health care costs. Few health systems implementing integrated care develop conceptual models and targeted measurement strategies a priori with an eye toward adoption, implementation, sustainment, and evaluation. This is a broad challenge in the field, which can make it difficult to disentangle why implementation is or is not successful.

Method

This paper discusses strategic implementation and evaluation planning for a pediatric integrated care program in a large health system. Our team developed a logic model, which defines resources and community characteristics, program components, evaluation activities, short-term activities, and intermediate and anticipated long-term patient-, clinician-, and practice-related outcomes. The model was designed based on research and stakeholder input to support strategic implementation and evaluation of the program. For each aspect of the logic model, a measurement battery was selected. Initial implementation data and intermediate outcomes from a pilot in five practices in a 30-practice pediatric primary care network are presented to illustrate how the logic model and evaluation plan have been used to guide the iterative process of program development.

Results

A total of 4,619 office visits were completed during the two years of the pilot. Primary care clinicians were highly satisfied with the integrated primary care program and provided feedback on ways to further improve the program. Members of the primary care team and behavioral health providers rated the program as being relatively well integrated into the practices after the second year of the pilot.

Conclusions

This logic model and evaluation plan provide a template for future projects integrating behavioral health services in non-specialty mental health settings, including pediatric primary care, and can be used broadly to provide structure to implementation and evaluation activities and promote replication of effective initiatives.

Social norms interventions to change clinical behaviour in health workers: a systematic review and meta-analysis

Background

A social norms intervention seeks to change the clinical behaviour of a target health worker by exposing them to the values, beliefs, attitudes or behaviours of a reference group or person. These low-cost interventions can be used to encourage health workers to follow recommended professional practice.

Objective

To summarise evidence on whether or not social norms interventions are effective in encouraging health worker behaviour change, and to identify the most effective social norms interventions.

Design

A systematic review and meta-analysis of randomised controlled trials.

Data sources

The following databases were searched on 24 July 2018: Ovid MEDLINE (1946 to week 2 July 2018), EMBASE (1974 to 3 July 2018), Cumulative Index to Nursing and Allied Health Literature (1937 to July 2018), British Nursing Index (2008 to July 2018), ISI Web of Science (1900 to present), PsycINFO (1806 to week 3 July 2018) and Cochrane trials (up to July 2018).

Participants

Health workers took part in the study.

Interventions

Behaviour change interventions based on social norms.

Outcome measures

Health worker clinical behaviour, for example prescribing (primary outcome), and patient health outcomes, for example blood test results (secondary), converted into a standardised mean difference.

Methods

Titles and abstracts were reviewed against the inclusion criteria to exclude any that were clearly ineligible. Two reviewers independently screened the remaining full texts to identify relevant papers. Two reviewers extracted data independently, coded for behaviour change techniques and assessed quality using the Cochrane risk-of-bias tool. We performed a meta-analysis and presented forest plots, stratified by behaviour change technique. Sources of variation were explored using metaregression and network meta-analysis.

Results

A total of 4428 abstracts were screened, 477 full texts were screened and findings were based on 106 studies. Most studies were in primary care or hospitals, targeting prescribing, ordering of tests and communication with patients. The interventions included social comparison (in which information is given on how peers behave) and credible source (which refers to communication from a well-respected person in support of the behaviour). Combined data suggested that interventions that included social norms components were associated with an improvement in health worker behaviour of 0.08 standardised mean differences (95% confidence interval 0.07 to 0.10 standardised mean differences) (n = 100 comparisons), and an improvement in patient outcomes of 0.17 standardised mean differences (95% confidence interval 0.14 to 0.20) (n = 14), on average. Heterogeneity was high, with an overall I 2 of 85.4% (primary) and 91.5% (secondary). Network meta-analysis suggested that three types of social norms intervention were most effective, on average, compared with control: credible source (0.30 standardised mean differences, 95% confidence interval 0.13 to 0.47); social comparison combined with social reward (0.39 standardised mean differences, 95% confidence interval 0.15 to 0.64); and social comparison combined with prompts and cues (0.33 standardised mean differences, 95% confidence interval 0.22 to 0.44).

Limitations

The large number of studies prevented us from requesting additional information from authors. The trials varied in design, context and setting, and we combined different types of outcome to provide an overall summary of evidence, resulting in a very heterogeneous review.

Conclusions

Social norms interventions are an effective method of changing clinical behaviour in a variety of health service contexts. Although the overall result was modest and very variable, there is the potential for social norms interventions to be scaled up to target the behaviour of a large population of health workers and resulting patient outcomes.

Future work

Development of optimised credible source and social comparison behaviour change interventions, including qualitative research on acceptability and feasibility.

Study registration

This study is registered as PROSPERO CRD42016045718.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 41. See the NIHR Journals Library website for further project information.

More Guidelines than states: variations in U.S. lead screening and management guidance and impacts on shareable CDS development

BACKGROUND

Pediatric lead exposure in the United States (U.S.) remains a preventable public health crisis. Shareable electronic clinical decision support (CDS) could improve lead screening and management. However, discrepancies between federal, state and local recommendations could present significant challenges for implementation.

METHODS

We identified publically available guidance on lead screening and management. We extracted definitions for elevated lead and recommendations for screening, follow-up, reporting, and management. We compared thresholds and level of obligation for management actions. Finally, we assessed the feasibility of development of shareable CDS.

RESULTS

We identified 54 guidance sources. States offered different definitions of elevated lead, and recommendations for screening, reporting, follow-up and management. Only 37 of 48 states providing guidance used the Center for Disease Control (CDC) definition for elevated lead. There were 17 distinct management actions. Guidance sources indicated an average of 5.5 management actions, but offered different criteria and levels of obligation for these actions. Despite differences, the recommendations were well-structured, actionable, and encodable, indicating shareable CDS is feasible.

CONCLUSION

Current variability across guidance poses challenges for clinicians. Developing shareable CDS is feasible and could improve pediatric lead screening and management. Shareable CDS would need to account for local variability in guidance.

Chronic Care for Attention-Deficit/Hyperactivity Disorder: Clinical Management from Childhood Through Adolescence

OBJECTIVE

Impairments generally persist when children diagnosed with attention-deficit/hyperactivity disorder (ADHD) mature into adolescence. To examine changes in ADHD care during the transition from childhood to adolescence, we conducted a retrospective, longitudinal cohort study of patients diagnosed with ADHD before the age of 10 years to assess changes from preadolescence through adolescence in (1) frequency by which primary care providers offer ADHD care to patients, (2) range of concerns assessed during patient encounters, and (3) treatments implemented or recommended.

METHODS

We identified patients from 3 practices included in a large primary care network who (1) were born between 1996 and 1997, (2) were diagnosed with ADHD before the age of 10 years, and (3) received primary care continuously from age 9 through late adolescence. Clinical care was compared among patients in preadolescence (age 9-11), early adolescence (age 12-14), and late adolescence (age 15-18).

RESULTS

Children diagnosed with ADHD before the age of 10 years were less likely to have a documented visit for ADHD during late adolescence (41% of patients) compared with preadolescence (63%, p < 0.001). Evidence of monitoring for depression, suicide, and substance abuse increased from preadolescence to adolescence (p < 0.001) and occurred in about 90% of adolescent patients. However, monitoring for risky sexual activity occurred in only about 50% of adolescents. Discussions of medication diversion and driver readiness were essentially not documented.

CONCLUSION

The findings raise concerns about how primary care providers manage adolescents with a history of ADHD. Improving monitoring of risky sexual behavior and driver readiness and providing patient education about medication diversion are needed.

Use of Technology to Promote Child Behavioral Health in the Context of Pediatric Care: A Scoping Review and Applications to Low- and Middle-Income Countries

Background: The burden of mental, neurological, and substance (MNS) disorders is greater in low- and middle-income countries (LMICs). The rapid growth of digital health (i.e., eHealth) approaches offer new solutions for transforming pediatric mental health services and have the potential to address multiple resource and system barriers. However, little work has been done in applying eHealth to promote young children's mental health in LMICs. It is also not clear how eHealth has been and might be applied to translating existing evidence-based practices/strategies (EBPs) to enable broader access to child mental health interventions and services. Methods: A scoping review was conducted to summarize current eHealth applications and evidence in child mental health. The review focuses on 1) providing an overview of existing eHealth applications, research methods, and effectiveness evidence in child mental health promotion (focused on children of 0-12 years of age) across diverse service contexts; and 2) drawing lessons learned from the existing research about eHealth design strategies and usability data in order to inform future eHealth design in LMICs. Results: Thirty-two (32) articles fitting our inclusion criteria were reviewed. The child mental health eHealth studies were grouped into three areas: i) eHealth interventions targeting families that promote child and family wellbeing; ii) eHealth for improving school mental health services (e.g., promote school staff's knowledge and management skills); and iii) eHealth for improving behavioral health care in the pediatric care system (e.g., promote use of integrated patient-portal and electronic decision support systems). Most eHealth studies have reported positive impacts. Although most pediatric eHealth studies were conducted in high-income countries, many eHealth design strategies can be adapted and modified to fit LMIC contexts. Most user-engagement strategies identified from high-income countries are also relevant for populations in LMICs. Conclusions: This review synthesizes patterns of eHealth use across a spectrum of individual/family and system level of eHealth interventions that can be applied to promote child mental health and strengthen mental health service systems. This review also summarizes critical lessons to guide future eHealth design and delivery models in LMICs. However, more research in testing combinations of eHealth strategies in LMICs is needed.

Sharing of ADHD Information between Parents and Teachers Using an EHR-Linked Application

BACKGROUND

Appropriate management of attention-deficit/hyperactivity disorder (ADHD) involves parents, clinicians, and teachers. Fragmentation of interventions between different settings can lead to suboptimal care and outcomes. Electronic systems can bridge gaps across settings. Our institution developed an email-based software to collect ADHD information from parents and teachers, which delivered data directly to the clinician within the electronic health record (EHR).

OBJECTIVE

We sought to adapt our institution's existing EHR-linked system for ADHD symptom monitoring to support communication between parents and teachers and then to assess child characteristics associated with sharing of ADHD information.

METHODS

We updated our software to support automated sharing of ADHD information between parents and teachers. Sharing was optional for parents but obligatory for teachers. We conducted a retrospective cohort study involving 590 patients at 31 primary care sites to evaluate a system for sharing of ADHD-specific health information between parents and teachers. We used multivariable logistic regression to estimate associations between child characteristics and parental sharing. We further investigated the association between child characteristics and viewing of survey results delivered through the electronic communication system.

RESULTS

Most parents (64%) elected to share survey results with teachers at the first opportunity and the vast majority (80%) elected to share all possible information. Parents who elected to share usually continue sharing at subsequent opportunities (89%). Younger child age and performance impairments were associated with increased likelihood of sharing. However, parents viewed only 16% of teacher submitted surveys and teachers only viewed 30% of surveys shared by parents.

CONCLUSION

This study demonstrates that electronic systems to capture ADHD information from parents and teachers can be adapted to support communication between them, and that parents are amenable to sharing ADHD information with teachers. However, strategies are needed to encourage viewing of shared information.