Assessing the validity of existing dental sealant quality measures

Development of quality measures to assess tooth decay outcomes from electronic health record data

OBJECTIVES

To develop outcomes of care quality measures derived from the dental electronic health record (EHR) to assess the occurrence and timely treatment of tooth decay.

METHODS

Quality measures were developed to assess whether decay was treated within 6 months and if new decay occurred in patients seen. Using EHR-derived data of the state of each tooth surface, algorithms compared the patient's teeth at different dates to determine if decay was treated or new decay had occurred. Manual chart reviews were conducted at three sites to validate the measures. The measures were implemented and scores were calculated for three sites over four calendar years, 2016 through 2019.

RESULTS

About 954 charts were manually reviewed for the timely treatment of tooth decay measure, with measure performance of sensitivity 97%, specificity 85%, positive predictive value (PPV) 91%, negative predictive value (NPV) 95%. About 739 charts were reviewed for new decay measure, with sensitivity 94%, specificity 99%, PPV 99%, and NPV 94%. Across all sites and years, 52.8% of patients with decay were fully treated within 6 months of diagnosis (n = 247,959). A total of 23.8% of patients experienced new decay, measured at an annual exam (n = 640,004).

CONCLUSION

Methods were developed and validated for assessing timely treatment of decay and occurrence of new decay derived from EHR data, creating effective outcome measures. These EHR-based quality measures produce accurate and reliable results that support efforts and advancement in quality assessment, quality improvement, patient care and research.

A generative adversarial inpainting network to enhance prediction of periodontal clinical attachment level

OBJECTIVES

Bone level as measured by clinical attachment levels (CAL) are critical findings that determine the diagnosis of periodontal disease. Deep learning algorithms are being used to determine CAL which aid in the diagnosis of periodontal disease. However, the limited field-of-view of bitewing x-rays poses a challenge for convolutional neural networks (CNN) because out-of-view anatomy cannot be directly considered. This study presents an inpainting algorithm using generative adversarial networks (GANs) coupled with partial convolutions to predict out-of-view anatomy to enhance CAL prediction accuracy.

METHODS

Retrospective purposive sampling of cases with healthy periodontium and diseased periodontium with bitewing and periapical radiographs and clinician recorded CAL were utilized. Data utilized was from July 1, 2016 through January 30, 2020. 80,326 images were used for training, 12,901 images were used for validation and 10,687 images were used to compare non-inpainted methods to inpainted methods for CAL predictions. Statistical analyses were mean bias error (MBE), mean absolute error (MAE) and Dunn's pairwise test comparing CAL at p=0.05.

RESULTS

Comparator p-values demonstrated statistically significant improvement in CAL prediction accuracy between corresponding inpainted and non-inpainted methods with MAE of 1.04 mm and 1.50 mm respectively. The Dunn's pairwise test indicated statistically significant improvement in CAL prediction accuracy between inpainted methods compared to their non-inpainted counterparts, with the best performing methods achieving a Dunn's pairwise value of -63.89.

CONCLUSIONS

This study demonstrates the superiority of using a generative adversarial inpainting network with partial convolutions to predict CAL from bitewing and periapical images.

CLINICAL SIGNIFICANCE

Artificial intelligence was developed and utilized to predict clinical attachment level compared to clinical measurements. A generative adversarial inpainting network with partial convolutions was developed, tested and validated to predict clinical attachment level. The inpainting approach was found to be superior to non-inpainted methods and within the 1mm clinician-determined measurement standard.

Measuring the quality of dental care among privately insured children in the United States

OBJECTIVE

To examine whether quality of dental care varies by age and over time and whether community-level characteristics explain these patterns.

DATA SOURCE

Deidentified medical and dental claims from a commercial insurer from January 2015 to December 2019.

STUDY DESIGN

A retrospective cohort study. The primary outcome was a composite quality score, derived from seven dental quality measures (DQMs), with higher values corresponding to better quality. Hierarchical regression models identified person- and zip code-level factors associated with the quality.

DATA COLLECTION/EXTRACTION METHODS

Continuously enrolled US dental insurance beneficiaries younger than 21 years of age.

PRINCIPAL FINDINGS

Quality was assessed for 4.88 million person-years covering 1.31 million persons. Overall quality slightly improved over time, mostly driven by substantial improvements among children aged 0-5 years by 0.153 points/year (95% confidence interval [CI]:0.151, 0.156). Quality was poorest and declined over time among adolescents with only 20.5% of DQMs met as compared to 42.6% among aged 0-5 years in 2019. Dental professional shortage, median household income, percentages of African Americans, unemployed, and less-educated populations at the zip code level were associated with the composite score.

CONCLUSION

Quality of dental care among adolescents remains low, and place of residence influenced the quality. Increasing the supply of dentists and oral health promotion strategies targeting adolescents and low-performing localities should be explored.

BigMouth: development and maintenance of a successful dental data repository

Few clinical datasets exist in dentistry to conduct secondary research. Hence, a novel dental data repository called BigMouth was developed, which has grown to include 11 academic institutions contributing Electronic Health Record data on over 4.5 million patients. The primary purpose for BigMouth is to serve as a high-quality resource for rapidly conducting oral health-related research. BigMouth allows for assessing the oral health status of a diverse US patient population; provides rationale and evidence for new oral health care delivery modes; and embraces the specific oral health research education mission. A data governance framework that encouraged data sharing while controlling contributed data was initially developed. This transformed over time into a mature framework, including a fee schedule for data requests and allowing access to researchers from noncontributing institutions. Adoption of BigMouth helps to foster new collaborations between clinical, epidemiological, statistical, and informatics experts and provides an additional venue for professional development.

Caries Risk Documentation And Prevention: eMeasures For Dental Electronic Health Records

BACKGROUND

Longitudinal patient level data available in the electronic health record (EHR) allows for the development, implementation, and validations of dental quality measures (eMeasures).

OBJECTIVE

We report the feasibility and validity of implementing two eMeasures. The eMeasures determined the proportion of patients receiving a caries risk assessment (eCRA) and corresponding appropriate risk-based preventative treatments for patients at elevated risk of caries (appropriateness of care [eAoC]) in two academic institutions and one accountable care organization, in the 2019 reporting year.

METHODS

Both eMeasures define the numerator and denominator beginning at the patient level, populations' specifications, and validated the automated queries. For eCRA, patients who completed a comprehensive or periodic oral evaluation formed the denominator, and patients of any age who received a CRA formed the numerator. The eAoC evaluated the proportion of patients at elevated caries risk who received the corresponding appropriate risk-based preventative treatments.

RESULTS

EHR automated queries identified in three sites 269,536 patients who met the inclusion criteria for receiving a CRA. The overall proportion of patients who received a CRA was 94.4% (eCRA). In eAoC, patients at elevated caries risk levels (moderate, high, or extreme) received fluoride preventive treatment ranging from 56 to 93.8%. For patients at high and extreme risk, antimicrobials were prescribed more frequently site 3 (80.6%) than sites 2 (16.7%) and 1 (2.9%).

CONCLUSION

Patient-level data available in the EHRs can be used to implement process-of-care dental eCRA and AoC, eAoC measures identify gaps in clinical practice. EHR-based measures can be useful in improving delivery of evidence-based preventative treatments to reduce risk, prevent tooth decay, and improve oral health.

Assessing the completeness of periodontal disease documentation in the EHR: a first step in measuring the quality of care

Background

Our objective was to measure the proportion of patients for which comprehensive periodontal charting, periodontal disease risk factors (diabetes status, tobacco use, and oral home care compliance), and periodontal diagnoses were documented in the electronic health record (EHR). We developed an EHR-based quality measure to assess how well four dental institutions documented periodontal disease-related information. An automated database script was developed and implemented in the EHR at each institution. The measure was validated by comparing the findings from the measure with a manual review of charts.

Results

The overall measure scores varied significantly across the four institutions (institution 1 = 20.47%, institution 2 = 0.97%, institution 3 = 22.27% institution 4 = 99.49%, p-value < 0.0001). The largest gaps in documentation were related to periodontal diagnoses and capturing oral homecare compliance. A random sample of 1224 charts were manually reviewed and showed excellent validity when compared with the data generated from the EHR-based measure (Sensitivity, Specificity, PPV, and NPV > 80%).

Conclusion

Our results demonstrate the feasibility of developing automated data extraction scripts using structured data from EHRs, and successfully implementing these to identify and measure the periodontal documentation completeness within and across different dental institutions.

Demonstration of payer readiness for value-based care in a fee-for-service environment: Measuring provider performance on sealant delivery

OBJECTIVES

Previously published sealant measures are not useful when applied to Medicaid claims data in states where dental services are carved out of managed care. A novel sealant measure was developed to assess the degree to which dental providers seal eligible teeth during preventive dental visits (PDVs) in an effort to ascertain if such a measure can be used to valuate provider performance, as condition of potential value-based care model implementation.

METHODS

A single-county feasibility study was conducted using Medicaid claims. A study cohort included children aged 8 years and enrolled 12 months during 2018. Prospective analysis was used to determine whether dental sealants were applied by the same dentists during PDVs or up to 9 months thereafter. Eligible teeth included first permanent molars. Teeth previously restored, sealed or missing were excluded. PDV was defined as any encounter with prophylaxis, fluoride treatment, or EPSDT. Claims were compared to public health surveillance for measurement validation.

RESULTS

Single-county results showed 11 percent of eligible teeth were sealed. Only 9 percent of dentists applied sealants to at least 40 percent of eligible teeth. Face validation of sealant rate was 23 percent Medicaid versus 36 percent Public Health. The former measures incidence and the latter prevalence with greater heterogeneity that included partially retained sealants.

CONCLUSIONS

A sealant measure that assesses provider adherence to sealant standards of care was produced. It has potential application for assessing performance of pediatric preventive services and informing value-based performance expectations.

Measuring sealant placement in children at the dental practice level

Background.

Although sealants are an established and recommended caries-preventive treatment, many children still fail to receive them. In addition, research has shown that existing measures underestimate care by overlooking the sealable potential of teeth before evaluating care. To address this, the authors designed and evaluated 3 novel dental electronic health record–based clinical quality measures that evaluate sealant care only after assessing the sealable potential of teeth.

Methods.

Measure I recorded the proportion of patients with sealable teeth who received sealants. Measure II recorded the proportion of patients who had at least 1 of their sealable teeth sealed. Measure III recorded the proportion of patients who received sealant on all of their sealable teeth.

Results.

On average, 48.1% of 6- through 9-year-old children received 1 or more sealants compared with 32.4% of 10- through 14-year-olds (measure I). The average measure score decreased for patients who received sealants for at least 1 of their sealable teeth (measure II) (43.2% for 6- through 9-year-olds and 28.4% for 10- through 14-year-olds). Fewer children received sealants on all eligible teeth (measure III) (35.5% of 6- through 9-year-olds and 21% of 10- through 14-year-olds received sealant on all eligible teeth). Among the 48.5% who were at elevated caries risk, the sealant rates were higher across all 3 measures.

Conclusions.

A valid and actionable practice-based sealant electronic measure that evaluates sealant treatment among the eligible population, both at the patient level and the tooth level, has been developed.

Practical Implications.

The measure developed in this work provides practices with patient-centered and actionable sealant quality measures that aim to improve oral health outcomes.

Increasing value, reducing waste: tailoring the application of dental sealants according to individual caries risk

OBJECTIVES

Despite a significant national investment in oral health, there is little understanding of the return in terms of quality. Value-based payments aim to refocus provider reimbursement based on the value created to the patient. Our objectives were to apply a set of dental quality measures to help determine the value of preventive dental care provided to children at two academic dental school clinics.

METHODS

We queried the institutional electronic health records (EHRs) for patients between the ages of 6-14 years with sealable first or second permanent molars, determined caries risk status, identified if dental sealants were placed, and finally if the teeth showed evidence of new caries experience. In order to determine the cost-effectiveness of EHR-based triage of applying dental sealants, we calculated the incremental cost-effectiveness ratio (ICER) for the dental quality measures supported sealing program.

RESULTS

Between the two academic sites, there were 6,155 unique children for a total of 12,302 eligible teeth without a sealant and 32,811 eligible teeth with a sealant. Teeth without a sealant were more likely to have decay (4.8 percent) than those with a sealant (1.7 percent). At both sites, patients with high caries risk were more likely to benefit from sealants compared to those patients with low risk.

CONCLUSION

Implementation of caries risk stratified fissure sealant quality measures demonstrates the potential for extracting better value in oral health care.

Leveraging Electronic Dental Record Data for Clinical Research in the National Dental PBRN Practices

OBJECTIVES

The aim of this study is to determine the feasibility of conducting clinical research using electronic dental record (EDR) data from U.S. solo and small-group general dental practices in the National Dental Practice-Based Research Network (network) and evaluate the data completeness and correctness before performing survival analyses of root canal treatment (RCT) and posterior composite restorations (PCR).

METHODS

Ninety-nine network general dentistry practices that used Dentrix or EagleSoft EDR shared de-identified data of patients who received PCR and/or RCT on permanent teeth through October 31, 2015. We evaluated the data completeness and correctness, summarized practice, and patient characteristics and summarized the two treatments by tooth type and arch location.

RESULTS

Eighty-two percent of practitioners were male, with a mean age of 49 and 22.4 years of clinical experience. The final dataset comprised 217,887 patients and 11,289,594 observations, with the observation period ranging from 0 to 37 years. Most patients (73%) were 18 to 64 years old; 56% were female. The data were nearly 100% complete. Eight percent of observations had incorrect data, such as incorrect tooth number or surface, primary teeth, supernumerary teeth, and tooth ranges, indicating multitooth procedures instead of PCR or RCT. Seventy-three percent of patients had dental insurance information; 27% lacked any insurance information. While gender was documented for all patients, race/ethnicity was missing in the dataset.

CONCLUSION

This study established the feasibility of using EDR data integrated from multiple distinct solo and small-group network practices for longitudinal studies to assess treatment outcomes. The results laid the groundwork for a learning health system that enables practitioners to learn about their patients' outcomes by using data from their own practice.

ADEA-ADEE Shaping the Future of Dental Education III: The impact of scientific technologies and discoveries on oral health globally

The central purpose of scientific research and emerging dental health technologies is to improve care for patients and achieve health equity. The Impact of Scientific Technologies and Discoveries on Oral Health Globally workshop conducted joint American Dental Education Association (ADEA) and the Association for Dental Education in Europe (ADEE) 2019 conference, Shaping the Future of Dental Education III, highlighted innovative technologies and scientific discoveries to support personalized dental care in an academic and clinical setting. The 2019 workshop built upon the new ideas and way forward identified in the 2017 ADEE-ADEA joint American Dental Education Association (ADEA) and the Association for Dental Education in Europe (ADEE) 2019 conference, Shaping the Future of Dental Education II held in London. During the most recent workshop the approach was to explore the "Teaching Clinic of the Future". Participants applied ideas proposed by keynote speakers, Dr. Walji and Dr. Vervoorn to educational models (Logic Model) in an ideal dental education setting. It is only through this continuous improvement of our use of scientific and technological advances that dental education will be able to convey to students the cognitive skills required to continually adapt to the changes that will affect them and consequently their patients throughout their career. This workshop was a valuable experience for highlighting opportunities and challenges for all stakeholders when aiming to incorporate new technologies to facilitate patient care and students' education.

Feasibility and usability of measuring receipt of sealants in 2 states

BACKGROUND

The authors examined the reliability and validity of the Dental Quality Alliance childhood sealant measure under actual use conditions in Texas and Florida. The 2 states provide care for almost 20% of children in Medicaid nationally.

METHODS

The authors used dental claims data to examine the reliability of the caries risk assessment component of the measure. They examined validity using a 3-year look-back period to identify children who were inaccurately included in the measure denominator as sealant eligible when they were not owing to already sealed, missing, or restored teeth.

RESULTS

The children identified at elevated risk varied between the states, with 85% at elevated risk in Texas and 39% in Florida in 2017. Different methods can be used to calculate risk, raising questions about reliability. In Texas, 31% of children included in the denominator were not eligible to receive sealants owing to already sealed, missing, or restored teeth. The magnitude of the underestimation increased with age, so by the time children were 9 years old, 40% were not measure eligible yet included in the denominator. Similar results were observed for Florida.

CONCLUSIONS

The authors propose eliminating the caries risk assessment requirement and incorporating a 3-year look-back period to identify already sealed, missing, or restored molars.

PRACTICAL IMPLICATIONS

The reliability and validity of the sealant measure needs to be enhanced. Measure misspecification in which children are not correctly identified as needing sealants can contribute to inaccurate development of quality improvement goals, performance improvement projects, or pay-for-quality programs.