Multi-modal deep learning for automated assembly of periapical radiographs

OBJECTIVES

Periapical radiographs are oftentimes taken in series to display all teeth present in the oral cavity. Our aim was to automatically assemble such a series of periapical radiographs into an anatomically correct status using a multi-modal deep learning model.

METHODS

4,707 periapical images from 387 patients (on average, 12 images per patient) were used. Radiographs were labeled according to their field of view and the dataset split into a training, validation, and test set, stratified by patient. In addition to the radiograph the timestamp of image generation was extracted and abstracted as follows: A matrix, containing the normalized timestamps of all images of a patient was constructed, representing the order in which images were taken, providing temporal context information to the deep learning model. Using the image data together with the time sequence data a multi-modal deep learning model consisting of two residual convolutional neural networks (ResNet-152 for image data, ResNet-50 for time data) was trained. Additionally, two uni-modal models were trained on image data and time data, respectively. A custom scoring technique was used to measure model performance.

RESULTS

Multi-modal deep learning outperformed both uni-modal image-based learning (p<0.001) and time-based learning (p<0.05). The multi-modal deep learning model predicted tooth labels with an F1-score, sensitivity and precision of 0.79, respectively, and an accuracy of 0.99. 37 out of 77 patient datasets were fully correctly assembled by multi-modal learning; in the remaining ones, usually only one image was incorrectly labeled.

CONCLUSIONS

Multi-modal modeling allowed automated assembly of periapical radiographs and outperformed both uni-modal models. Dental machine learning models can benefit from additional data modalities.

CLINICAL SIGNIFICANCE

Like humans, deep learning models may profit from multiple data sources for decision-making. We demonstrate how multi-modal learning can assist assembling periapical radiographs into an anatomically correct status. Multi-modal learning should be considered for more complex tasks, as clinically a wealth of data is usually available and could be leveraged.

Identification of Dental Implant Systems Using a Large-Scale Multicenter Data Set

This study aimed to evaluate the efficacy of deep learning (DL) for the identification and classification of various types of dental implant systems (DISs) using a large-scale multicenter data set. We also compared the classification accuracy of DL and dental professionals. The data set, which was collected from 5 college dental hospitals and 10 private dental clinics, contained 37,442 (24.8%) periapical and 113,291 (75.2%) panoramic radiographic images and consisted of a total of 10 manufacturers and 25 different types of DISs. The classification accuracy of DL was evaluated using a pretrained and modified ResNet-50 architecture, and comparison of accuracy performance and reading time between DL and dental professionals was conducted using a self-reported questionnaire. When comparing the accuracy performance for classification of DISs, DL (accuracy: 82.0%; 95% confidence interval [CI], 75.9%-87.0%) outperformed most of the participants (mean accuracy: 23.5% ± 18.5%; 95% CI, 18.5%-32.3%), including dentists specialized (mean accuracy: 43.3% ± 20.4%; 95% CI, 12.7%-56.2%) and not specialized (mean accuracy: 16.8% ± 9.0%; 95% CI, 12.8%-20.9%) in implantology. In addition, DL tends to require lesser reading and classification time (4.5 min) than dentists who specialized (75.6 ± 31.0 min; 95% CI, 13.1-78.4) and did not specialize (91.3 ± 38.3 min; 95% CI, 74.1-108.6) in implantology. DL achieved reliable outcomes in the identification and classification of various types of DISs, and the classification accuracy performance of DL was significantly superior to that of specialized or nonspecialized dental professionals. DL as a decision support aid can be successfully used for the identification and classification of DISs encountered in clinical practice.

Artificial Intelligence for Caries Detection: Value of Data and Information

If increasing practitioners’ diagnostic accuracy, medical artificial intelligence (AI) may lead to better treatment decisions at lower costs, while uncertainty remains around the resulting cost-effectiveness. In the present study, we assessed how enlarging the data set used for training an AI for caries detection on bitewings affects cost-effectiveness and also determined the value of information by reducing the uncertainty around other input parameters (namely, the costs of AI and the population’s caries risk profile). We employed a convolutional neural network and trained it on 10%, 25%, 50%, or 100% of a labeled data set containing 29,011 teeth without and 19,760 teeth with caries lesions stemming from bitewing radiographs. We employed an established health economic modeling and analytical framework to quantify cost-effectiveness and value of information. We adopted a mixed public–private payer perspective in German health care; the health outcome was tooth retention years. A Markov model, allowing to follow posterior teeth over the lifetime of an initially 12-y-old individual, and Monte Carlo microsimulations were employed. With an increasing amount of data used to train the AI sensitivity and specificity increased nonlinearly, increasing the data set from 10% to 25% had the largest impact on accuracy and, consequently, cost-effectiveness. In the base-case scenario, AI was more effective (tooth retention for a mean [2.5%–97.5%] 62.8 [59.2–65.5] y) and less costly (378 [284–499] euros) than dentists without AI (60.4 [55.8–64.4] y; 419 [270–593] euros), with considerable uncertainty. The economic value of reducing the uncertainty around AI’s accuracy or costs was limited, while information on the population’s risk profile was more relevant. When developing dental AI, informed choices about the data set size may be recommended, and research toward individualized application of AI for caries detection seems warranted to optimize cost-effectiveness.

Federated Learning in Dentistry: Chances and Challenges

Building performant and robust artificial intelligence (AI)-based applications for dentistry requires large and high-quality data sets, which usually reside in distributed data silos from multiple sources (e.g., different clinical institutes). Collaborative efforts are limited as privacy constraints forbid direct sharing across the borders of these data silos. Federated learning is a scalable and privacy-preserving framework for collaborative training of AI models without data sharing, where instead the knowledge is exchanged in form of wisdom learned from the data. This article aims at introducing the established concept of federated learning together with chances and challenges to foster collaboration on AI-based applications within the dental research community.

Towards Trustworthy AI in Dentistry

Medical and dental artificial intelligence (AI) require the trust of both users and recipients of the AI to enhance implementation, acceptability, reach, and maintenance. Standardization is one strategy to generate such trust, with quality standards pushing for improvements in AI and reliable quality in a number of attributes. In the present brief review, we summarize ongoing activities from research and standardization that contribute to the trustworthiness of medical and, specifically, dental AI and discuss the role of standardization and some of its key elements. Furthermore, we discuss how explainable AI methods can support the development of trustworthy AI models in dentistry. In particular, we demonstrate the practical benefits of using explainable AI on the use case of caries prediction on near-infrared light transillumination images.

Benchmarking Deep Learning Models for Tooth Structure Segmentation

A wide range of deep learning (DL) architectures with varying depths are available, with developers usually choosing one or a few of them for their specific task in a nonsystematic way. Benchmarking (i.e., the systematic comparison of state-of-the art architectures on a specific task) may provide guidance in the model development process and may allow developers to make better decisions. However, comprehensive benchmarking has not been performed in dentistry yet. We aimed to benchmark a range of architecture designs for 1 specific, exemplary case: tooth structure segmentation on dental bitewing radiographs. We built 72 models for tooth structure (enamel, dentin, pulp, fillings, crowns) segmentation by combining 6 different DL network architectures (U-Net, U-Net++, Feature Pyramid Networks, LinkNet, Pyramid Scene Parsing Network, Mask Attention Network) with 12 encoders from 3 different encoder families (ResNet, VGG, DenseNet) of varying depth (e.g., VGG13, VGG16, VGG19). On each model design, 3 initialization strategies (ImageNet, CheXpert, random initialization) were applied, resulting overall into 216 trained models, which were trained up to 200 epochs with the Adam optimizer (learning rate = 0.0001) and a batch size of 32. Our data set consisted of 1,625 human-annotated dental bitewing radiographs. We used a 5-fold cross-validation scheme and quantified model performances primarily by the F1-score. Initialization with ImageNet or CheXpert weights significantly outperformed random initialization (P < 0.05). Deeper and more complex models did not necessarily perform better than less complex alternatives. VGG-based models were more robust across model configurations, while more complex models (e.g., from the ResNet family) achieved peak performances. In conclusion, initializing models with pretrained weights may be recommended when training models for dental radiographic analysis. Less complex model architectures may be competitive alternatives if computational resources and training time are restricting factors. Models developed and found superior on nondental data sets may not show this behavior for dental domain-specific tasks.

Data Dentistry: How Data Are Changing Clinical Care and Research

Data are a key resource for modern societies and expected to improve quality, accessibility, affordability, safety, and equity of health care. Dental care and research are currently transforming into what we term data dentistry, with 3 main applications: 1) medical data analysis uses deep learning, allowing one to master unprecedented amounts of data (language, speech, imagery) and put them to productive use. 2) Data-enriched clinical care integrates data from individual (e.g., demographic, social, clinical and omics data, consumer data), setting (e.g., geospatial, environmental, provider-related data), and systems level (payer or regulatory data to characterize input, throughput, output, and outcomes of health care) to provide a comprehensive and continuous real-time assessment of biologic perturbations, individual behaviors, and context. Such care may contribute to a deeper understanding of health and disease and a more precise, personalized, predictive, and preventive care. 3) Data for research include open research data and data sharing, allowing one to appraise, benchmark, pool, replicate, and reuse data. Concerns and confidence into data-driven applications, stakeholders’ and system’s capabilities, and lack of data standardization and harmonization currently limit the development and implementation of data dentistry. Aspects of bias and data-user interaction require attention. Action items for the dental community circle around increasing data availability, refinement, and usage; demonstrating safety, value, and usefulness of applications; educating the dental workforce and consumers; providing performant and standardized infrastructure and processes; and incentivizing and adopting open data and data sharing.

A prospective, multi-center, practice-based cohort study on all-ceramic crowns

OBJECTIVES

The aim of this prospective, multi-center, practice-based cohort study was to analyze factors associated with the success of all-ceramic crowns.

METHODS

All-ceramic crowns placed in a practice-based research network ([Ceramic Success Analysis, AG Keramik) were analyzed. Data from 1254 patients with (mostly in-office CAD/CAM) all-ceramic crowns placed by 101 dentists being followed up for more than 5 years were evaluated. At the last follow-up visit crowns were considered as successful (not failed) if they were sufficient, whereas crowns were considered as survived (not lost) if they were still in function. Multi-level Cox proportional hazards models were used to evaluate the association between a range of predictors and time of success or survival.

RESULTS

Within a mean follow-up period (SD) of 7.2(2)years [maximum:15years] 776 crowns were considered successful (annual failure rate[AFR]:8.4%) and 1041 crowns survived (AFR:4.9%). The presence of a post in endodontically treated teeth resulted in a risk for failure 2.7 times lower than that of restorations without a post (95%CI:1.4-5.0;p = 0.002). Regarding the restorative material and adhesive technique, hybrid composite ceramics and single-step adhesives showed a 3.4 and 2.2 times higher failure rate than feldspathic porcelain and multi-step adhesives, respectively (p < 0.001). Use of an oxygen-blocking gel as well as an EVA instrument resulted in a 1.5-1.8 times higher failure rate than their non-use (p ≤ 0.001).

SIGNIFICANCE

After up to 15years AFR were rather high for all-ceramic crowns. Operative factors, but no patient- or tooth-level factors were significantly associated with failure. The study was registered in the German Clinical Trials Register (DRKS-ID: DRKS00020271).

Better Reporting of Studies on Artificial Intelligence: CONSORT-AI and Beyond

An increasing number of studies on artificial intelligence (AI) are published in the dental and oral sciences. The reporting, but also further aspects of these studies, suffer from a range of limitations. Standards towards reporting, like the recently published Consolidated Standards of Reporting Trials (CONSORT)-AI extension can help to improve studies in this emerging field, and the Journal of Dental Research (JDR) encourages authors, reviewers, and readers to adhere to these standards. Notably, though, a wide range of aspects beyond reporting, located along various steps of the AI lifecycle, should be considered when conceiving, conducting, reporting, or evaluating studies on AI in dentistry.

Cost-effectiveness of Artificial Intelligence for Proximal Caries Detection

Artificial intelligence (AI) can assist dentists in image assessment, for example, caries detection. The wider health and cost impact of employing AI for dental diagnostics has not yet been evaluated. We compared the cost-effectiveness of proximal caries detection on bitewing radiographs with versus without AI. U-Net, a fully convolutional neural network, had been trained, validated, and tested on 3,293, 252, and 141 bitewing radiographs, respectively, on which 4 experienced dentists had marked carious lesions (reference test). Lesions were stratified for initial lesions (E1/E2/D1, presumed noncavitated, receiving caries infiltration if detected) and advanced lesions (D2/D3, presumed cavitated, receiving restorative care if detected). A Markov model was used to simulate the consequences of true- and false-positive and true- and false-negative detections, as well as the subsequent decisions over the lifetime of patients. A German mixed-payers perspective was adopted. Our health outcome was tooth retention years. Costs were measured in 2020 euro. Monte-Carlo microsimulations and univariate and probabilistic sensitivity analyses were conducted. The incremental cost-effectiveness ratio (ICER) and the cost-effectiveness acceptability at different willingness-to-pay thresholds were quantified. AI showed an accuracy of 0.80; dentists’ mean accuracy was significantly lower at 0.71 (minimum–maximum: 0.61–0.78, P < 0.05). AI was significantly more sensitive than dentists (0.75 vs. 0.36 [0.19–0.65]; P = 0.006), while its specificity was not significantly lower (0.83 vs. 0.91 [0.69–0.98]; P > 0.05). In the base-case scenario, AI was more effective (tooth retention for a mean 64 [2.5%–97.5%: 61–65] y) and less costly (298 [244–367] euro) than assessment without AI (62 [59–64] y; 322 [257–394] euro). The ICER was −13.9 euro/y (i.e., AI saved money at higher effectiveness). In the majority (>77%) of all cases, AI was less costly and more effective. Applying AI for caries detection is likely to be cost-effective, mainly as fewer lesions remain undetected. Notably, this cost-effectiveness requires dentists to manage detected early lesions nonrestoratively.

Artificial Intelligence in Dentistry: Chances and Challenges

The term “artificial intelligence” (AI) refers to the idea of machines being capable of performing human tasks. A subdomain of AI is machine learning (ML), which “learns” intrinsic statistical patterns in data to eventually cast predictions on unseen data. Deep learning is a ML technique using multi-layer mathematical operations for learning and inferring on complex data like imagery. This succinct narrative review describes the application, limitations and possible future of AI-based dental diagnostics, treatment planning, and conduct, for example, image analysis, prediction making, record keeping, as well as dental research and discovery. AI-based applications will streamline care, relieving the dental workforce from laborious routine tasks, increasing health at lower costs for a broader population, and eventually facilitate personalized, predictive, preventive, and participatory dentistry. However, AI solutions have not by large entered routine dental practice, mainly due to 1) limited data availability, accessibility, structure, and comprehensiveness, 2) lacking methodological rigor and standards in their development, 3) and practical questions around the value and usefulness of these solutions, but also ethics and responsibility. Any AI application in dentistry should demonstrate tangible value by, for example, improving access to and quality of care, increasing efficiency and safety of services, empowering and enabling patients, supporting medical research, or increasing sustainability. Individual privacy, rights, and autonomy need to be put front and center; a shift from centralized to distributed/federated learning may address this while improving scalability and robustness. Lastly, trustworthiness into, and generalizability of, dental AI solutions need to be guaranteed; the implementation of continuous human oversight and standards grounded in evidence-based dentistry should be expected. Methods to visualize, interpret, and explain the logic behind AI solutions will contribute (“explainable AI”). Dental education will need to accompany the introduction of clinical AI solutions by fostering digital literacy in the future dental workforce.

Can We Predict Usage of Dental Services? An Analysis from Germany 2000 to 2015

OBJECTIVES

We aimed to predict the usage of dental services in Germany from 2000 to 2015 based on epidemiologic and demographic data, and to compare these predictions against claims within the statutory health insurance.

METHODS

Indicators for operative (number of coronally decayed or filled teeth, root surface caries lesions, and fillings), prosthetic (number of missing teeth), and periodontal treatment needs (number of teeth with probing pocket depths (PPDs) ≥ 4 mm) from nationally representative German Oral Health Studies (1997, 2005, 2014) were cross-sectionally interpolated across age and time, and combined with year- and age-specific population estimates. These, as well as the number of children eligible for individual preventive services (aged 6 to 17 y), were adjusted for age- and time-specific insurance status and services' utilization to yield predicted usage of operative, prosthetic, periodontal, and preventive services. Cumulative annual usage in these 4 services groups were compared against aggregations of a total of 24 claims positions from the statutory German health insurance.

RESULTS

Morbidity, utilization, and demography were highly dynamic across age groups and over time. Despite improvements of individual oral health, predicted usage of dental services did not decrease over time, but increased mainly due to usage shifts from younger (shrinking) to older (growing) age groups. Predicted usage of operative services increased between 2000 and 2015 (from 52 million to 56 million, +7.8%); predictions largely agreed with claimed services (root mean square error [RMSE] 1.9 million services, error range -4.6/+3.8%). Prosthetic services increased (from 2.4 million to 2.6 million, +11.9%), with near perfect agreement to claimed data [RMSE 0.1 million services, error range -8.3/+3.9%]). Periodontal services also increased (from 21 million to 27 million, +25.9%; RMSE 5.2 million services, error range +21.9/+36.5%), as did preventive services (from 22 million to 27 million, +20.4%; RMSE 3 million, error range -13.7/-4.7%).

CONCLUSION

Predicting dental services seems viable when accounting for the joint dynamics of morbidity, utilization, and demographics.

KNOWLEDGE TRANSFER STATEMENT

Based on epidemiologic and demographic data, predicting usage of certain dental services is viable when accounting for the dynamics of morbidity, utilization, and demographics.

Evaluating Modeling and Validation Strategies for Tooth Loss

Prediction models learn patterns from available data (training) and are then validated on new data (testing). Prediction modeling is increasingly common in dental research. We aimed to evaluate how different model development and validation steps affect the predictive performance of tooth loss prediction models of patients with periodontitis. Two independent cohorts (627 patients, 11,651 teeth) were followed over a mean ± SD 18.2 ± 5.6 y (Kiel cohort) and 6.6 ± 2.9 y (Greifswald cohort). Tooth loss and 10 patient- and tooth-level predictors were recorded. The impact of different model development and validation steps was evaluated: 1) model complexity (logistic regression, recursive partitioning, random forest, extreme gradient boosting), 2) sample size (full data set or 10%, 25%, or 75% of cases dropped at random), 3) prediction periods (maximum 10, 15, or 20 y or uncensored), and 4) validation schemes (internal or external by centers/time). Tooth loss was generally a rare event (880 teeth were lost). All models showed limited sensitivity but high specificity. Patients' age and tooth loss at baseline as well as probing pocket depths showed high variable importance. More complex models (random forest, extreme gradient boosting) had no consistent advantages over simpler ones (logistic regression, recursive partitioning). Internal validation (in sample) overestimated the predictive power (area under the curve up to 0.90), while external validation (out of sample) found lower areas under the curve (range 0.62 to 0.82). Reducing the sample size decreased the predictive power, particularly for more complex models. Censoring the prediction period had only limited impact. When the model was trained in one period and tested in another, model outcomes were similar to the base case, indicating temporal validation as a valid option. No model showed higher accuracy than the no-information rate. In conclusion, none of the developed models would be useful in a clinical setting, despite high accuracy. During modeling, rigorous development and external validation should be applied and reported accordingly.

Cost-effectiveness of the Hall Technique in a Randomized Trial

Clinical and patient-reported outcomes were reported for carious primary molars treated with the Hall technique (HT) as compared with conventional carious tissue removal and restorations (i.e., conventional restoration [CR]) in a 5-y randomized controlled practice-based trial in Scotland. We interrogated this data set further to investigate the cost-effectiveness of HT versus CR. A total of 132 children who had 2 matched occlusal/occlusal-proximal carious lesions in primary molars ( n = 264 teeth) were randomly allocated to HT or CR, provided by 17 general dental practitioners. Molars were followed up for a mean 5 y. A societal perspective was taken for the economic analysis. Direct dental treatment costs were estimated from a Scottish NHS perspective (an NHS England perspective was taken for a sensitivity analysis). Initial, maintenance, and retreatment costs, including rerestorations, endodontic treatments, and extractions, were estimated with fee items. Indirect/opportunity costs were estimated with time and travel costs from a UK perspective. The primary outcome was tooth survival. Secondary outcomes included 1) not having pain or needing endodontic treatments/extractions and 2) not needing rerestorations. Cost-effectiveness and acceptability were estimated from bootstrapped samples. Significantly more molars in HT survived (99%, 95% CI: 98% to 100%) than in CR (92%; 87% to 97%). Also, the proportion of molars retained without pain or requiring endodontic treatment/extraction was significantly higher in HT than CR. In the base case analysis (NHS Scotland perspective), cumulative direct dental treatment costs (Great British pound [GBP]) of HT were 24 GBP (95% CI: 23 to 25); costs for CR were 29 (17 to 46). From an NHS England perspective, the cost advantage of HT (29 GBP; 95% CI: 25 to 34) over CR (107; 86 to 127) was more pronounced. Indirect/opportunity costs were significantly lower for HT (8 GBP; 95% CI: 7 to 9) than CR (19; 16 to 23). Total cumulative costs were significantly lower for HT (32 GBP; 95% CI: 31 to 34) than CR (49; 34 to 69). Based on a long-term practice-based trial, HT was more cost-effective than CR with HT retained for longer and experiencing less complications at lower costs.

Amalgam Alternatives: Cost-Effectiveness and Value of Information Analysis

We aimed to assess the cost-effectiveness of amalgam alternatives—namely, incrementally placed composites (IComp), composites placed in bulk (BComp), and glass ionomer cements (GIC). In a sensitivity analysis, we also included composite inlays (CompI) and incrementally placed bulk-fills (IBComp). Moreover, the value of information (VOI) regarding the effectiveness of all strategies was determined. A mixed public-private-payer perspective in the context of Germany was adopted. Bayesian network meta-analyses were performed to yield effectiveness estimates (relative risk [RR] of failure). A 3-surfaced restoration on a permanent molar in initially 30-y-old patients was followed over patients’ lifetime using a Markov model. Restorative and endodontic complications were modeled; our outcome parameter was the years of tooth retention. Costs were derived from insurance fee items. Monte Carlo microsimulations were used to estimate cost-effectiveness, cost-effectiveness acceptability, and VOI. Initially, BComp/GIC were less costly (110.11 euros) than IComp (146.82 euros) but also more prone to failures (RRs [95% credible intervals (CrI)] were 1.6 [0.8 to 3.4] for BComp and 1.3 [0.5 to 5.6] for GIC). When following patients over their lifetime, IComp was most effective (mean [SD], 41.9 [1] years) and least costly (2,076 [135] euros), hence dominating both BComp (40.5 [1] years; 2,284 [126] euros) and GIC (41.2 years; 2,177 [126] euros) in 90% of simulations. Eliminating the uncertainty around the effectiveness of the strategies was worth 3.99 euros per restoration, translating into annual economic savings of 87.8 million euros for payers. Including CompI and IBComp into our analyses had only a minimal impact, and our findings were robust in further sensitivity analyses. In conclusion, the initial savings by BComp/GIC compared with IComp are very likely to be compensated by the higher risk of failures and costs for retreatments. CompI and IBComp do not seem cost-effective. All alternatives are likely to be inferior to amalgam. The VOI was considerable, and future studies may yield significant economic benefits.