Improved interobserver variation after training of doctors in the Neer system

A Comprehensive Radiologic Review of Shoulder Girdle Trauma

Radiologic knowledge of different fracture patterns involving the shoulder girdle is an important tool to generate clinically relevant reports, identify concomitant injuries, guide management decisions, and predict and minimize complications, such as nonunion, osteoarthritis, osteonecrosis, and hardware failure. Complex unstable injuries like scapulothoracic dissociation can also occur because of shoulder girdle trauma. Management options may vary from conservative to surgical, depending on the fracture type and patient factors. Injuries around the shoulder girdle can involve the glenohumeral articulation, scapula, superior shoulder suspensory complex, acromioclavicular joints, and scapulothoracic articulation.

Influence of training on dorsal tangential radiographic view to detect screw protrusion after anterior plating of the distal radius: a cadaveric study

In this cadaveric study of anterior plating of the distal radius, we aimed to determine the interobserver agreement and diagnostic performance for detecting dorsally protruding screws using the dorsal tangential radiographic view before and after specific training. Without prior instruction, 13 observers interpreted the dorsal tangential view of cadaveric specimens, in which anterior radial plates were placed. After seeing a training video on the dorsal tangential view, they repeated the task. Though we found that accuracy and interobserver agreement was lower than described in some other clinical series, training led to statistically significant improvements of (1) the interobserver agreement on the decision to exchange screws, (2) the self-confidence of the surgeon in obtaining adequate views, and (3) the number of fluoroscopic images required to obtain these views. After training, the number of protruding screws missed was reduced by 36%, but 7% of dorsally protruding screws was still missed.

Does 3D CT reconstruction help the surgeon to preoperatively assess calcaneal fractures?

BACKGROUND

With the advent of 3D volume rendered CT scans, more information is potentially available to aid the surgeon in complex calcaneal fractures. The primary aim was to determine if there is a difference in inter-observer and intra-observer reliability of 3D CT reconstructions compared to 2D CTs of calcaneus fractures based on classification, identification of specific fracture characteristics and proposed treatment. It is hypothesized that the correlation will be greater between experienced surgeons and trainees when using 3D CT. As a secondary aim, the authors wished to investigate the surgeons' satisfaction with 3D CT.

PATIENTS AND METHODS

There were six raters, split into 2 groups: high and low surgical experience. Both 2D and 3D scans (10 patients) were reviewed by each rater by filling out a validated questionnaire. This sitting was over a period of six months and all scans were sent separately and randomly by our research coordinator. This process was repeated after a four-week break.

RESULTS

The overall evaluation and satisfaction of CT scans of calcaneal fractures was improved by the additional use of volume rendered 3D images. Inexperienced surgeons benefited more than experienced surgeons in terms of reliabilities with fracture configuration. 3D CTs were more likely to change the surgeons' operative strategy. The consistency of deciding on the fracture classification and satisfaction was more uniform between all the raters when 3D CT was used.

CONCLUSION

Inter-observer and intra-observer reliability of 3D CT reconstructions compared to 2D CTs of calcaneus fractures was greater based on classification, identification of specific fracture characteristics and proposed treatment. Inexperienced surgeons were more consistent when interpreting the scans using 3D CTs (improved inter-observer reliability) and were always more satisfied using the 3D CTs. As there is no extra cost or radiation exposure, we propose that 3D CTs may be valuable with preoperative teaching and planning.

Defining the fracture population in a pragmatic multicentre randomised controlled trial: PROFHER and the Neer classification of proximal humeral fractures

OBJECTIVES

Accurate characterisation of fractures is essential in fracture management trials. However, this is often hampered by poor inter-observer agreement. This article describes the practicalities of defining the fracture population, based on the Neer classification, within a pragmatic multicentre randomised controlled trial in which surgical treatment was compared with non-surgical treatment in adults with displaced fractures of the proximal humerus involving the surgical neck.

METHODS

The trial manual illustrated the Neer classification of proximal humeral fractures. However, in addition to surgical neck displacement, surgeons assessing patient eligibility reported on whether either or both of the tuberosities were involved. Anonymised electronic versions of baseline radiographs were sought for all 250 trial participants. A protocol, data collection tool and training presentation were developed and tested in a pilot study. These were then used in a formal assessment and classification of the trial fractures by two independent senior orthopaedic shoulder trauma surgeons.

RESULTS

Two or more baseline radiographic views were obtained for each participant. The independent raters confirmed that all fractures would have been considered for surgery in contemporaneous practice. A full description of the fracture population based on the Neer classification was obtained. The agreement between the categorisation at baseline (tuberosity involvement) and Neer classification as assessed by the two raters was only fair (kappa 0.29). However, this disparity did not appear to affect trial findings, specifically in terms of influencing the effect of treatment on the primary outcome of the trial.

CONCLUSIONS

A key reporting requirement, namely the description of the fracture population, was achieved within the context of a pragmatic multicentre randomised clinical trial. This article provides important guidance for researchers designing similar trials on fracture management.Cite this article: H. H. G. Handoll, S. D. Brealey, L. Jefferson, A. Keding, A. J. Brooksbank, A. J. Johnstone, J. J. Candal-Couto, A. Rangan. Defining the fracture population in a pragmatic multicentre randomised controlled trial: PROFHER and the Neer classification of proximal humeral fractures.Bone Joint Res 2016;5:481-489. DOI: 10.1302/2046-3758.510.BJR-2016-0132.R1.

The Impact of Three-Dimensional CT Imaging on Intraobserver and Interobserver Reliability of Proximal Humeral Fracture Classifications and Treatment Recommendations

BACKGROUND

The classification systems for fractures of the proximal part of the humerus provide low interobserver and intraobserver reliability when radiographs or two-dimensional computed tomography scans are used. The purpose of this investigation was to determine whether the use of three-dimensional computed tomography scans could improve interobserver and intraobserver reliability of AO/OTA (Arbeitsgemeinschaft für Osteosynthesefragen/Orthopaedic Trauma Association) and Neer classifications and treatment recommendations.

METHODS

Two trauma surgeons, one shoulder surgeon, two senior orthopaedic residents, and two junior orthopaedic residents reviewed the radiographs and two and three-dimensional computed tomography scans of forty fractures of the proximal part of the humerus. Each imaging modality was reviewed in isolation, and fractures were classified according to the Neer and AO/OTA classifications and treatment recommendations were provided. This process was repeated for intraobserver analysis. Interobserver agreement was calculated within and between levels of training for each classification and treatment recommendation with respect to radiographs and two and three-dimensional computed tomography scans.

RESULTS

Among attending orthopaedic surgeons and senior residents, the use of three-dimensional computed tomography did not improve agreement compared with the use of two-dimensional computed tomography for the Neer classification based on planes, the AO/OTA classification, or the treatment recommendation, but it did improve agreement among junior residents. Comparing between levels of training, three-dimensional computed tomography increased agreement only between junior residents and more experienced reviewers for the Neer classification based on planes and for the AO/OTA classification but not for the treatment recommendation. Intraobserver agreement for each reviewer for classification and treatment ranged from slight to fair and was not improved through the use of three-dimensional computed tomography.

CONCLUSIONS

In this investigation, the use of three-dimensional computed tomography imaging did not offer improved interobserver and intraobserver agreement compared with the use of two-dimensional computed tomography imaging with regard to classification and treatment of fractures of the proximal part of the humerus, except among reviewers with limited clinical experience.

LEVEL OF EVIDENCE

Diagnostic Level III. See Instructions for Authors for a complete description of levels of evidence.

Interobserver reliability of classification and characterization of proximal humeral fractures: a comparison of two and three-dimensional CT

BACKGROUND

Interobserver reliability for the classification of proximal humeral fractures is limited. The aim of this study was to test the null hypothesis that interobserver reliability of the AO classification of proximal humeral fractures, the preferred treatment, and fracture characteristics is the same for two-dimensional (2-D) and three-dimensional (3-D) computed tomography (CT).

METHODS

Members of the Science of Variation Group--fully trained practicing orthopaedic and trauma surgeons from around the world--were randomized to evaluate radiographs and either 2-D CT or 3-D CT images of fifteen proximal humeral fractures via a web-based survey and respond to the following four questions: (1) Is the greater tuberosity displaced? (2) Is the humeral head split? (3) Is the arterial supply compromised? (4) Is the glenohumeral joint dislocated? They also classified the fracture according to the AO system and indicated their preferred treatment of the fracture (operative or nonoperative). Agreement among observers was assessed with use of the multirater kappa (κ) measure.

RESULTS

Interobserver reliability of the AO classification, fracture characteristics, and preferred treatment generally ranged from "slight" to "fair." A few small but statistically significant differences were found. Observers randomized to the 2-D CT group had slightly but significantly better agreement on displacement of the greater tuberosity (κ = 0.35 compared with 0.30, p < 0.001) and on the AO classification (κ = 0.18 compared with 0.17, p = 0.018). A subgroup analysis of the AO classification results revealed that shoulder and elbow surgeons, orthopaedic trauma surgeons, and surgeons in the United States had slightly greater reliability on 2-D CT, whereas surgeons in practice for ten years or less and surgeons from other subspecialties had slightly greater reliability on 3-D CT.

CONCLUSIONS

Proximal humeral fracture classifications may be helpful conceptually, but they have poor interobserver reliability even when 3-D rather than 2-D CT is utilized. This may contribute to the similarly poor interobserver reliability that was observed for selection of the treatment for proximal humeral fractures. The lack of a reliable classification confounds efforts to compare the outcomes of treatment methods among different clinical trials and reports.

Inter-Observer, Intra-Observer and Intra-Individual Reliability of Uroflowmetry Tests in Aged Men: A Generalizability Theory Approach

OBJECTIVES

To evaluate the inter-observer, intra-observer and intra-individual reliability of uroflowmetry and post-void residual urine (PVR) tests in adult men.

METHODS

Healthy volunteers aged over 40 years were enrolled. Every participant underwent two sets of uroflowmetry and PVR tests with a 2-week interval between the tests. The uroflowmetry tests were interpreted by four urologists independently. Uroflowmetry curves were classified as bell-shaped, bell-shaped with tail, obstructive, restrictive, staccato, interrupted and tower-shaped and scored from 1 (highly abnormal) to 5 (absolutely normal). The agreements between the observers, interpretations and tests within individuals were analyzed using kappa statistics and intraclass correlation coefficients. Generalizability theory with decision analysis was used to determine how many observers, tests, and interpretations were needed to obtain an acceptable reliability (> 0.80).

RESULTS

Of 108 volunteers, we randomly selected the uroflowmetry results from 25 participants for the evaluation of reliability. The mean age of the studied adults was 55.3 years. The intra-individual and intra-observer reliability on uroflowmetry tests ranged from good to very good. However, the inter-observer reliability on normalcy and specific type of flow pattern were relatively lower. In generalizability theory, three observers were needed to obtain an acceptable reliability on normalcy of uroflow pattern if the patient underwent uroflowmetry tests twice with one observation.

CONCLUSIONS

The intra-individual and intra-observer reliability on uroflowmetry tests were good while the inter-observer reliability was relatively lower. To improve inter-observer reliability, the definition of uroflowmetry should be clarified by the International Continence Society.

Inter- and intra-observer variability of the Crowe and Hartofilakidis classification systems for congenital hip disease in adults

Our study evaluated the reliability of the Crowe and Hartofilakidis classification systems for developmental dysplasia of the hip in adults. The anteroposterior radiographs of the pelvis of 145 patients with 209 osteoarthritic hips were examined twice by three experienced hip surgeons from three European countries and the abnormal hips were rated using both classifications. The inter- and intra-observer agreement was calculated. Interobserver reliability was evaluated using weighted and unweighted kappa coefficients and for the Crowe classification, among the three pairs there was a minimum kappa coefficient with linear weighting of 0.90 for observers A and C and a maximum kappa coefficient of 0.92 for observers B and C. For the Hartofilakidis classification, the minimum kappa value was 0.85 for observers A and B, and the maximum value was 0.93 for observers B and C. With regard to intra-observer reliability, the kappa coefficients with linear weighting between the two evaluations of the same observer ranged between 0.86 and 0.95 for the Crowe classification and between 0.80 and 0.93 for the Hartofilakidis classification. The reliability of both systems was substantial to almost perfect both for serial measurements by individual readers and between different readers, although the information offered was dissimilar.