Interobserver and Intraobserver Agreement are Unsatisfactory When Determining Abstract Study Design and Level of Evidence

BACKGROUND

Understanding differences between types of study design (SD) and level of evidence (LOE) are important when selecting research for presentation or publication and determining its potential clinical impact. The purpose of this study was to evaluate interobserver and intraobserver reliability when assigning LOE and SD as well as quantify the impact of a commonly used reference aid on these assessments.

METHODS

Thirty-six accepted abstracts from the Pediatric Orthopaedic Society of North America (POSNA) 2021 annual meeting were selected for this study. Thirteen reviewers from the POSNA Evidence-Based Practice Committee were asked to determine LOE and SD for each abstract, first without any assistance or resources. Four weeks later, abstracts were reviewed again with the guidance of the Journal of Bone and Joint Surgery (JBJS) LOE chart, which is adapted from the Oxford Centre for Evidence-Based Medicine. Interobserver and intraobserver reliability were calculated using Fleiss' kappa statistic (k). χ2 analysis was used to compare the rate of SD-LOE mismatch between the first and second round of reviews.

RESULTS

Interobserver reliability for LOE improved slightly from fair (k=0.28) to moderate (k=0.43) with use of the JBJS chart. There was better agreement with increasing LOE, with the most frequent disagreement between levels 3 and 4. Interobserver reliability for SD was fair for both rounds 1 (k=0.29) and 2 (k=0.37). Similar to LOE, there was better agreement with stronger SD. Intraobserver reliability was widely variable for both LOE and SD (k=0.10 to 0.92 for both). When matching a selected SD to its associated LOE, the overall rate of correct concordance was 82% in round 1 and 92% in round 2 (P<0.001).

CONCLUSION

Interobserver reliability for LOE and SD was fair to moderate at best, even among experienced reviewers. Use of the JBJS/Oxford chart mildly improved agreement on LOE and resulted in less SD-LOE mismatch, but did not affect agreement on SD.

LEVEL OF EVIDENCE

Level II.

Characteristics of Pyogenic Musculoskeletal Infections in Older Children and Adolescents

The characteristics and clinical consequences of pyogenic bone and joint infections in older children and adolescents have received little attention. This study evaluated the presentation and complications of musculoskeletal infections involving the pelvis and extremities in children older than 10 years. Thirty patients 10 to 17 years old (mean, 12.7 years old) were treated for musculoskeletal infections. Mean time to diagnosis was 9.2 days. Prior to correct diagnosis, 83% were assessed by at least 1 outpatient provider. At the time of admission, 55% were weight bearing and 93% were afebrile. Twenty-eight percent had a multifocal infection. More than one-third had serious medical complications or orthopedic sequelae; compared with patients without complications, this group had a significantly higher admission C-reactive protein and longer hospital stay. Symptoms of musculoskeletal infection common among young children may be absent in adolescents. Axial imaging is recommended to identify adjacent or multifocal disease. The Kocher criteria are less sensitive for septic hip arthritis in the adolescent population. Prompt recognition and treatment are critical to avoid medical and musculoskeletal complications. [Orthopedics. 2020;43(4):e291-e298.].

Evaluation of a Novel Method for Determining Transverse Plane Pelvic Obliquity

STUDY DESIGN

This was a retrospective review of neuromuscular scoliosis radiographs evaluating interobserver and intra-observer error for a novel method of transverse plane pelvic obliquity.

OBJECTIVES

To evaluate the utility of a previously described method by Lucas et al. of determining transverse plane pelvic obliquity using standard radiographs in patients with cerebral palsy and neuromuscular scoliosis.

SUMMARY OF BACKGROUND DATA

Evaluation of pelvic obliquity in the transverse plane has not been thoroughly studied. The pelvis has been noted to function as intercalary vertebra in neuromuscular scoliosis, resulting in marked obliquity in all 3 planes.

METHODS

Forty radiographs were chosen from 10 patients with cerebral palsy and neuromuscular scoliosis who had had a posterior spine arthrodesis and Galveston spino-pelvic fixation. Four observers independently examined the radiographs at different levels of training on 2 dates 1 week apart. Measurements recorded by each observer were described by Lucas et al.: E (the distance measured on lateral radiographs between the ilium at the inferior part of the sacro-iliac joint and the lateral edge of the anterior superior iliac spine), F_R and F_L (the coronal plane linear distance between the same 2 landmarks, measured from a posteroanterior radiograph, where F was measured for both the left (F_L) and right (F_R) sides of the pelvis, respectively), and β (the transverse plane rotation of the pelvis). Reproducibility of the measurements were analyzed using the concordance correlation coefficient (CCC). A CCC of 0.80 or higher was considered excellent agreement.

RESULTS

The CCC between the first and second sets of measurements was lowest for E and highest for the calculated β, although none of the CCC calculations was statistically significant, demonstrating poor agreement.

CONCLUSIONS

The ability to reliably measure and calculate the degree of transverse plane rotation by radiographs in cerebral palsy patients with spino-pelvic deformity by the method described by Lucas et al. is poor, likely because of difficulty in consistently identify pelvic landmarks.

The modified Dunn procedure for unstable slipped capital femoral epiphysis: a multicenter perspective

BACKGROUND

The modified Dunn procedure has rapidly gained popularity as a treatment for unstable slipped capital femoral epiphysis (SCFE), but limited data exist regarding its safety and efficacy. The purpose of this study was to present results and complications following this procedure in a large multicenter series.

METHODS

We reviewed the outcomes of all patients who had been treated with the modified Dunn procedure by five surgeons from separate tertiary-care institutions. All slipped capital femoral epiphyses were defined as unstable according to the Loder criteria. Patients with less than one year of follow-up and those with an underlying endocrinopathy or syndrome were excluded. All surgical procedures were performed by pediatric orthopaedic surgeons who had specific training in the modified Dunn procedure. Operative reports, outpatient records, and follow-up radiographs were used to determine the demographic information, type of fixation, final slip angle, presence of osteonecrosis, and any additional complications. Standardized surveys were administered to determine the pain level (0 to 10 scale), satisfaction (0 to 100 scale), function (modified Harris hip score, 0 to 91 scale), and activity level (UCLA [University of California Los Angeles] activity score, 0 to 10 scale) at time of the most recent follow-up.

RESULTS

Twenty-seven patients (twenty-seven hips) with a mean of 22.3 months (range, twelve to forty-eight months) of follow-up met the inclusion criteria. Four patients (15%) had broken implants at three to eighteen weeks after surgery and required revision fixation. Seven patients (26%) developed osteonecrosis at a mean of 21.4 weeks (range, ten to thirty-nine weeks), with each surgeon having at least one case of osteonecrosis. The mean slip angle at the time of the most recent follow-up was 6° (95% confidence interval, 2° to 11°). Patients who did not develop osteonecrosis had significantly better clinical results compared with those who developed osteonecrosis, as demonstrated by a lower mean pain score (0.3 compared with 3.1, p = 0.002), higher level of satisfaction (97.1 compared with 65.8, p = 0.001), higher modified Harris hip score (88.0 compared with 60.0, p = 0.001), and higher UCLA activity score (9.3 compared with 5.9, p = 0.031).

CONCLUSIONS

This largest reported series of unstable slipped capital femoral epiphyses treated with the modified Dunn procedure demonstrated that the procedure is capable of restoring anatomy and preserving function after a slip but that implant complications and osteonecrosis can and do occur postoperatively.

Total hip arthroplasty in adolescents with severe hip arthropathy and dysplasia associated with camptodactyly-arthropathy-coxa vara-pericarditis syndrome

Camptodactyly-arthropathy-coxa vara-pericarditis (CACP) syndrome is a rare disorder. Patients with this syndrome experience early symptomatic arthropathy of the hips. We report a case of adolescent siblings with bilateral arthropathy associated with CACP syndrome in which total hip arthroplasty was performed as treatment of severe associated disability. Postoperative Harris Hip Scores for patient 1 were 86 for the right at 18 months and 96 for the left at 12 months. Postoperative Harris Hip Score at 6 months for patient 2 was 53; however, he had good range of motion and lacked deformity. Based on our limited experience and the limited available clinical data, we feel that total hip arthroplasty is a reasonable treatment option for adolescents with debilitating hip arthropathy associated with CACP syndrome.

Utility of electrodiagnostic testing and computed tomography myelography in the preoperative evaluation of neonatal brachial plexus palsy

OBJECT

The rate of neonatal brachial plexus palsy (NBPP) remains 0.4%-4% despite improvements in perinatal care. Among affected children, the extent of brachial plexus palsy differs greatly, as does the prognosis. Controversial elements in management include indications and timing of nerve repair as well as type of reconstruction in patients in whom function will ultimately not be recovered without surgical intervention. Differentiating preganglionic (avulsion) from postganglionic (rupture) lesions is critical because preganglionic lesions cannot spontaneously recover motor function. Distinguishing between these lesions at initial presentation based on clinical examination alone can be difficult in infants. The purpose of the present study was to determine the sensitivity of preoperative electrodiagnostic studies (EDSs) and CT myelography (CTM) in determining the presence of nerve root rupture and avulsions in infants with NBPP.

METHODS

After receiving institutional review board approval, the authors conducted a retrospective review of patients referred to the Neonatal Brachial Plexus Program between 2007 and 2010. Inclusion criteria included children who underwent brachial plexus exploration following preoperative EDSs and CTM. The CTM scans were interpreted by a staff neuroradiologist, EDSs were conducted by a single physiatrist, and intraoperative findings were recorded by the operating neurosurgeon. The findings from the preoperative EDSs and CTM were then compared with intraoperative findings. The sensitivities and 95% confidence intervals were determined to evaluate performance accuracy of each preoperative measure.

RESULTS

Twenty-one patients (8 male amd 13 female) met inclusion criteria for this study. The sensitivity of EDSs and CTM for detecting a postganglionic rupture was 92.8% (CI 0.841-0.969) and 58.3% (CI 0.420-0.729), respectively. The sensitivity for EDSs and CTM for preganglionic nerve root avulsion was 27.8% (CI 0.125-0.509) and 72.2% (CI 0.491-0.875), respectively. In cases in which both CTM and EDSs gave concordant results, the sensitivity for both modalities combined was 50.0% (CI 0.237-0.763) for avulsion and 80.8% (CI 0.621-0.915) for rupture. Overall, EDSs were most useful in identifying ruptures, particularly in the upper plexus, whereas CTM was most sensitive in identifying avulsions in the lower plexus.

CONCLUSIONS

Knowledge of the spinal nerve integrity is critical for early management of patients with NBPP. Surgical management, in the form of nerve repair/reconstruction, and optimal prognostication of NBPP depend on the accurate diagnosis of the level and type of lesion. Both EDSs and CTM scans must always be interpreted in the context of a comprehensive evaluation of the patient. They provide supplemental information (in addition to the physical examination) for early detection of nerve root rupture and avulsion injuries, aiding surgical decision making and preoperative planning for NBPP. Continued advances in imaging, EDSs, and microsurgical nerve repair techniques will allow surgeons to achieve greater success for functional recovery in management of NBPP.

Facet violation with the placement of percutaneous pedicle screws

STUDY DESIGN

Independent review and classification of therapeutic procedures performed on cadavers by surgeons blinded to purpose of study.

OBJECTIVE

The objective of this study is to determine the rate of facet violation with the placement of percutaneous pedicle screws.

SUMMARY OF BACKGROUND DATA

Improvements in percutaneous instrumentation and fluoroscopic imaging have led to a resurgence of percutaneous pedicle screw insertion in lumbar spine surgery in an attempt to minimize many of the complications associated with open techniques of pedicle screw placement. Rates of pedicle breech and neurologic injury resulting from percutaneous insertion are reportedly similar to those of open techniques. Postoperative pain because of impingement and instability is believed to result from violation of the facet capsule or facet joint. To the authors' knowledge, however, the rate of facet injury associated with the placement of percutaneous pedicle screws is unreported in the literature.

METHODS

Percutaneous pedicle screw placement was performed on 4 cadaveric specimens by 4 certified orthopedic surgeons who had clinical experience in the procedure and who were blinded to the study's purpose. The surgeons were instructed to place pedicle screws from L1-S1 using their preferred clinical techniques and a 5.5-mm screw system with which they were all familiar. All surgeons utilized 1 OEC C-arm for fluoroscopic imaging. After insertion, 2 independent spine surgeons each reviewed and classified the placement of all facet screws.

RESULTS

A total of 48 screws were inserted and classified. The placement of 28 screws (58%) resulted in violation of facet articulation, with 8 of these screws being intra-articular. Interobserver reliability of the classification system was 100%.

CONCLUSION

Percutaneous pedicle screw placement may result in a high rate of facet violation. Facet injury can be reliability classified and therefore, perhaps, easily prevented.

Bilateral unstable slipped capital femoral epiphysis: a look at risk factors

Unstable slipped capital femoral epiphysis can have disastrous complications including osteonecrosis and chondrolysis. It has been shown that 20% to 80% of patients may develop a contralateral slip ≤18 months after diagnosis. The purpose of this article is to report and characterize patients who developed bilateral unstable slips. After Institutional Review Board approval, the patients included were only those with bilateral unstable slipped capital femoral epiphyses. A minimum 2-year follow-up was required. Seven patients, all female, were included in the study, with an average age of 11.4 years at the time of their first slips. The interval between slips averaged 127 days (range, 0-245 days). All but 1 patient presented with a severe slip. The second slip was also severe in 3 patients and less severe in 4 patients. The triradiate cartilage was open in 3 patients. Two patients required corrective osteotomies. Chondrolysis developed in 2 patients with no osteonecrosis reported. The incidence of bilateral unstable slips ranged from 4% to 20% of all unstable slipped capital femoral epiphyses based on our findings. Skeletal immaturity was not a risk factor. The surgeon must be vigilant for the possibility of bilateral slips. The family must be instructed on precautions patients must take while recuperating from unstable slipped capital femoral epiphyses. Contralateral fixation of the unaffected side may be warranted in patients with initial severe unstable slipped capital femoral epiphyses to prevent this condition.

Vertical expandable prosthetic titanium rib device insertion: does it improve pulmonary function?

PURPOSE

Vertical expandable prosthetic titanium rib (VEPTR) insertion and expansion has been advocated to increase thoracic volume and pulmonary function in patients with thoracic insufficiency syndrome. We reviewed our experience with VEPTR implantation to determine if lung function and growth is augmented, to determine the children's functional status, and if the scoliosis is controlled.

METHODS

From 2006 to 2010, 29 insertions and 57 expansions were performed in 26 patients at our institution. Demographic data were reviewed in conjunction with complications, scoliosis angles, pulmonary function tests (PFTs), and computed tomography-guided 3D reconstructions to determine lung volumes; and quality of life scores were determined using a modified Scoliosis Research Society (SRS) questionnaire preoperatively and postoperatively. The groups were also stratified by age (because of lung growth potential), disease (congenital or infantile scoliosis, Jeune syndrome, neuromuscular, other structural thoracic disorders), and sex. Analyses using SPSS (SPSS, Chicago, Ill) were performed with P < .05 considered significant.

RESULTS

Each patient underwent 3.03 ± 1.8 surgeries, spending 0.97 ± 1.8 days in the intensive care unit and 4.41 ± 6 days in the hospital for each procedure. Mean age was 90.7 ± 41 months. Of the 36 complications, most were because of infection (12), half requiring operative repair (hardware removal). The average PFT percent predicted values for forced expiratory volume in 1 second, forced vital capacity, and RV were 54.6 ± 22, 58.1 ± 24, and 145.3 ± 112, respectively, preoperatively and 51.8 ± 20, 55.9 ± 20, and 105.6 ± 31, respectively, postoperatively. The lung volumes measured by computed tomography when corrected for age do not increase significantly postoperatively. The mean Cobb measurement for the preoperative major curves was 64.7° and postoperatively was 46.1° for those curves measured preoperatively, for a 29% curve improvement. All postoperative curves had a mean of 56.4° and 58.1° at final follow-up, a 3% curve increase. The SRS scores for patients remained unchanged and no statistical difference was seen from preoperative to postoperative values. No statistically significant difference was seen in complications, PFT (forced expiratory volume in 1 second, forced vital capacity, RV), lung volumes, scoliosis angles, and SRS scores between sex, age, and disease categories.

CONCLUSION

There was mild improvement in scoliosis angles but no improvement in lung function and volume. Scoliosis Research Society scores indicate that the children have near normal function both before and after VEPTR placement. Pulmonary function, lung volume, and patient subjective assessments did not increase dramatically after VEPTR placement, although scoliosis angles improved.

Lyme disease presenting as popliteal cyst in children

Lyme disease is the most common tick-borne disease in North America. Our review of the literature found few reports of Lyme disease presented in the orthopaedic literature. However, Lyme disease presenting as a popliteal cyst, with or without rupture, is rarely reported. We present 4 cases of Lyme disease that initially presented to our pediatric orthopaedic clinic for treatment of a popliteal cyst. The early diagnosis and treatment of Lyme disease may help prevent the often-devastating long-term sequelae of Lyme disease. The goal of this article is to increase the awareness of Lyme disease presenting in children as a popliteal cyst.