Interobserver and intraobserver agreement of three-dimensionally printed models for the classification of proximal humeral fractures

Improving Patient Understanding of Femoroacetabular Impingement Syndrome With Three-Dimensional Models

INTRODUCTION

Three-dimensional (3D) printed models may help patients understand complex anatomic pathologies such as femoroacetabular impingement syndrome (FAIS). We aimed to assess patient understanding and satisfaction when using 3D printed models compared with standard imaging modalities for discussion of FAIS diagnosis and surgical plan.

METHODS

A consecutive series of 76 new patients with FAIS (37 patients in the 3D model cohort and 39 in the control cohort) from a single surgeon's clinic were educated using imaging and representative 3D printed models of FAI or imaging without models (control). Patients received a voluntary post-visit questionnaire that evaluated their understanding of the diagnosis, surgical plan, and visit satisfaction.

RESULTS

Patients in the 3D model cohort reported a significantly higher mean understanding of FAIS (90.0 ± 11.5 versus 79.8 ± 14.9 out of 100; P = 0.001) and surgery (89.5 ± 11.6 versus 81.0 ± 14.5; P = 0.01) compared with the control cohort. Both groups reported high levels of satisfaction with the visit.

CONCLUSION

In this study, the use of 3D printed models in clinic visits with patients with FAIS improved patients' perceived understanding of diagnosis and surgical treatment.

The role of three dimension computed tomography in Lisfranc injury diagnosis

PURPOSE

To evaluate the intra/inter observer and diagnostic reliability of 3-dimensional (3D) model reconstruction computed tomography (CT) compared to the traditional two-dimensional (2D) CT when evaluating Lisfranc (LF) injuries.

METHODS

A retrospective study was performed on CT studies of patients with clinically suspected LF injuries examined in the emergency department at medium size medical center. Each CT study was evaluated for metatarsal fractures and subluxations employing both standard 2D and 3D CT model reconstruction. Four orthopaedists, 2 senior and 2 residents, were assigned to review and evaluate each CT. Each CT study was reviewed twice by each reader in a randomized order. Descriptive statistics were calculated for all measured variables. The intra-observer and inter-observer agreement Kappa coefficients were calculated to evaluate reliability and reproducibility between and within readers for each modality.

RESULTS

The study included 44 patients. Median age was 41.4 years (interquartile range, 23-58). The intra-observer and inter-observer reliability was good (intra-observer; 3D Kappa; 0.76 and 2D Kappa 0.73 p<0.001, inter-observer; 3D Kappa 0.68, 2D Kappa 0.63 and combined 2D and 3D kappa 0.68, p<0.001). Three dimensional CT was found to be more sensitive than 2D CT, specifically when evaluating for second metatarsal dislocation (sensitivity: 70% vs 47%). Combined evaluation of 2D and 3D CT, greatly improved sensitivity rate to 85.7%. In terms of fracture diagnosis, combined evaluation of 2D and 3D CT showed higher sensitivity and specificity rates as compared to 2D or 3D alone.

CONCLUSION

The employment of 3D CT in LF injury diagnosis, in isolation and in combination with 2D CT, seems to improve the diagnostic accuracy and reliability between and within observers compared to 2D CT alone.

Three-dimensional printing models increase inter-rater agreement for classification and treatment of proximal humerus fractures

BACKGROUND

Proximal humerus fractures (PHF) are frequent, however, several studies show low inter-rater agreement in the diagnosis and treatment of these injuries. Differences are usually related to the experience of the evaluators and/or the diagnostic methods used. This study was designed to investigate the hypothesis that shoulder surgeons and diagnostic imaging specialists using 3D printing models and shoulder CT scans in assessing proximal humerus fractures.

METHODS

We obtained 75 tomographic exams of PHF to print three-dimensional models. After, two shoulder surgeons and two specialists in musculoskeletal imaging diagnostics analyzed CT scans and 3D models according to the Neer and AO/OTA group classification and suggested a treatment recommendation for each fracture based on the two diagnostic methods.

RESULTS

The classification agreement for PHF using 3D printing models among the 4 specialists was moderate (global k = 0.470 and 0.544, respectively for AO/OTA and Neer classification) and higher than the CT classification agreement (global k = 0.436 and 0.464, respectively for AO/OTA and Neer). The inter-rater agreement between the two shoulder surgeons were substantial. For the AO/OTA classification, the inter-rater agreement using 3D printing models was higher (k = 0.700) than observed for CT (k = 0.631). For Neer classification,  inter-rater agreement with 3D models was similarly higher (k = 0.784) than CT images (k = 0.620). On the other hand, the inter-rater agreement between the two specialists in diagnostic imaging was moderate. In the AO/OTA classification, the agreement using CT was higher (k = 0.532) than using 3D printing models (k = 0.443), while for Neer classification, the agreement was similar for both 3D models (k = 0.478) and CT images (k = 0.421). Finally, the inter-rater agreement in the treatment of PHF by the 2 surgeons was higher for both classifications using 3D printing models (AO/OTA-k = 0.818 for 3D models and k = 0.537 for CT images). For Neer classification, we saw k = 0.727 for 3D printing models and k = 0.651 for CT images.

CONCLUSION

The insights from this diagnostic pilot study imply that for shoulder surgeons, 3D printing models improved the diagnostic agreement, especially the treatment indication for PHF compared to CT for both AO/OTA and Neer classifications On the other hand, for specialists in diagnostic imaging, the use of 3D printing models was similar to CT scans for diagnostic agreement using both classifications.

TRIAL REGISTRATION

Brazil Platform under no. CAAE 12273519.7.0000.5505.

3D-printed Handheld Models Do Not Improve Recognition of Specific Characteristics and Patterns of Three-part and Four-part Proximal Humerus Fractures

BACKGROUND

Reliably recognizing the overall pattern and specific characteristics of proximal humerus fractures may aid in surgical decision-making. With conventional onscreen imaging modalities, there is considerable and undesired interobserver variability, even when observers receive training in the application of the classification systems used. It is unclear whether three-dimensional (3D) models, which now can be fabricated with desktop printers at relatively little cost, can decrease interobserver variability in fracture classification.

QUESTIONS/PURPOSES

Do 3D-printed handheld models of proximal humerus fractures improve agreement among residents and attending surgeons regarding (1) specific fracture characteristics and (2) patterns according to the Neer and Hertel classification systems?

METHODS

Plain radiographs, as well as two-dimensional (2D) and 3D CT images, were collected from 20 patients (aged 18 years or older) who sustained a three-part or four-part proximal humerus fracture treated at a Level I trauma center between 2015 and 2019. The included images were chosen to comprise images from patients whose fractures were considered as difficult-to-classify, displaced fractures. Consequently, the images were assessed for eight fracture characteristics and categorized according to the Neer and Hertel classifications by four orthopaedic residents and four attending orthopaedic surgeons during two separate sessions. In the first session, the assessment was performed with conventional onscreen imaging (radiographs and 2D and 3D CT images). In the second session, 3D-printed handheld models were used for assessment, while onscreen imaging was also available. Although proximal humerus classifications such as the Neer classification have, in the past, been shown to have low interobserver reliability, we theorized that by receiving direct tactile and visual feedback from 3D-printed handheld fracture models, clinicians would be able to recognize the complex 3D aspects of classification systems reliably. Interobserver agreement was determined with the multirater Fleiss kappa and scored according to the categorical rating by Landis and Koch. To determine whether there was a difference between the two sessions, we calculated the delta (difference in the) kappa value with 95% confidence intervals and a two-tailed p value. Post hoc power analysis revealed that with the current sample size, a delta kappa value of 0.40 could be detected with 80% power at alpha = 0.05.

RESULTS

Using 3D-printed models in addition to conventional imaging did not improve interobserver agreement of the following fracture characteristics: more than 2 mm medial hinge displacement, more than 8 mm metaphyseal extension, surgical neck fracture, anatomic neck fracture, displacement of the humeral head, more than 10 mm lesser tuberosity displacement, and more than 10 mm greater tuberosity displacement. Agreement regarding the presence of a humeral head-splitting fracture was improved but only to a level that was insufficient for clinical or scientific use (fair to substantial, delta kappa = 0.33 [95% CI 0.02 to 0.64]). Assessing 3D-printed handheld models adjunct to onscreen conventional imaging did not improve the interobserver agreement for pattern recognition according to Neer (delta kappa = 0.02 [95% CI -0.11 to 0.07]) and Hertel (delta kappa = 0.01 [95% CI -0.11 to 0.08]). There were no differences between residents and attending surgeons in terms of whether 3D models helped them classify the fractures, but there were few differences to identify fracture characteristics. However, none of the identified differences improved to almost perfect agreement (kappa value above 0.80), so even those few differences are unlikely to be clinically useful.

CONCLUSION

Using 3D-printed handheld fracture models in addition to conventional onscreen imaging of three-part and four-part proximal humerus fractures does not improve agreement among residents and attending surgeons on specific fracture characteristics and patterns. Therefore, we do not recommend that clinicians expend the time and costs needed to create these models if the goal is to classify or describe patients' fracture characteristics or pattern, since doing so is unlikely to improve clinicians' abilities to select treatment or estimate prognosis.

LEVEL OF EVIDENCE

Level III, diagnostic study.