Reliability assessment of the OMERACT whole-body magnetic resonance imaging scoring system for juvenile idiopathic arthritis

Inter-reader reliability of a new scoring system for evaluating joint inflammation and enthesitis in whole body MRI (WBMRI) in juvenile idiopathic arthritis was tested. The scoring system grades 732 item-region combinations of bone marrow and soft tissue changes for commonly involved joints and entheseal sites. Five radiologists rated 17 WBMRI scans through an online rating platform. Item-wise reliability was calculated for 117 items with non-zero scores in >10 % of readings. Interquartile ranges of the five-reader Kappa reliability coefficients were 0.58-0.73 (range: 0.36-0.88) for the joints, 0.65-0.81 (range: 0.39-0.95) for the entheses, and 0.62-0.75 (range: 0.60-0.76) for chronic nonbacterial osteomyelitis-like lesions.

Systematic calibration reduces sources of variability for the preliminary OMERACT juvenile idiopathic arthritis MRI- sacroiliac joint score (OMERACT JAMRIS-SIJ)

OBJECTIVE

To determine whether systematic calibration enhances scoring proficiency of the OMERACT juvenile idiopathic arthritis MRI-Sacroiliac Joint score (JAMRIS-SIJ) and whether contrast-enhancement enhances its performance.

METHODS

MRI SIJ scans of 50 cases with juvenile spondyloarthritis were scored by 7 raters after calibration with 3 different knowledge transfer tools.

RESULTS

Calibrated readers achieved greater reliability for scoring certain inflammatory and structural lesions. Sensitivity and reliability for scoring inflammatory lesions was greater on fluid-sensitive compared to contrast-enhanced sequences.

CONCLUSION

Systematic calibration should be implemented prior to the use of JAMRIS-SIJ for clinical trials. It is unlikely that contrast-enhanced MRI will improve the performance of this method.

Developing standards for MRI evaluation of joints in children with juvenile idiopathic arthritis utilizing the temporomandibular joint as a model

The treatment of a patient with juvenile idiopathic arthritis (JIA) is best monitored with standardized and validated tools to measure joint changes over time. Treatment approaches are best indicated if the clinicians are aware of the structural status of the joint at a given time, especially in anatomically deep joints for which clinical assessment is limited. Magnetic resonance imaging (MRI) is of utmost importance for assessment of deep joints and extra-articular soft tissue of the entire body for which ultrasound may be suboptimal. Because the distinction between pathologic and physiologic joint changes on MRI is key for proper diagnosis and treatment of patients with arthropathies, a comprehensive standardized approach is needed to effectively measure outcomes of growing joints of children with JIA. Such an approach is essential for both clinical assessment and to conduct clinical trials in patients with JIA treated in different centers around the world. To meet this need, several international imaging collaborative research groups have been developing MRI scales over the past years, including the MRI in JIA (JAMRI) special interest group within the Outcome Measures in Rheumatology (OMERACT) research network. This manuscript reviews the efforts of the OMERACT JAMRI working group to generate and validate pediatric MRI scoring systems for different joints in children with JIA that can have ubiquitous utilization anywhere in the world. In particular, it describes the different steps of development and validation of an MRI scale using the TMJ as a model.

Somatic symptoms, pain, catastrophizing and the association with disability among children with heritable connective tissue disorders

The aim of the present study was to investigate the nature and prevalence of nonspecific somatic symptoms, pain and catastrophizing in children with Heritable Connective Tissue Disorders (HCTD), and to determine their association with disability. This observational, multicenter study included 127 children, aged 4-18 years, with Marfan syndrome (MFS) (59%), Loeys-Dietz syndrome (LDS) (8%), Ehlers-Danlos syndromes (EDS) (12%) and hypermobile Ehlers-Danlos syndrome (hEDS) (23%). The assessments included the Children's Somatization Inventory or parent proxy (CSI, PCSI), pain visual-analogue scale (VAS), SUPERKIDZ body diagram, Pain Catastrophizing Scale Child or parent proxy (PCS-C, PCS-P) and Childhood Health Assessment Questionnaire (CHAQ-30). Data from children aged ≥8 years were compared to normative data. In children ≥ 8 years (n = 90), pain was present in 59%, with a median of 4 (IQR = 3-9) pain areas. Compared to normative data, the HCTD group reported significantly higher on the CSI (p ≤ 0.001, d = 0.85), VAS pain intensity (p ≤ 0.001, d = 1.22) and CHAQ-30 (p ≤ 0.001, d = 1.16) and lower on the PCS-C (p = 0.017, d = -0.82) and PCS-P (p ≤ 0.001, d = -0.49). The intensity of nonspecific somatic symptoms and pain explained 45% of the variance in disability (r²  = 0.45 F(2,48) = 19.70, p ≤ 0.001). In children ≤ 7 years (n = 37),   pain was present in 35% with a median of 5(IQR = 1-13) pain areas. The mean(SD) VAS scores for pain intensity was 1.5(2.9). Functional disability was moderately correlated to the number of pain areas (r = 0.56, p ≤ 0.001), intensity of nonspecific somatic symptoms (r = 0.63, p ≤ 0.001) and pain (r = 0.83, p ≤ 0.001). In conclusion, this study supports the need for comprehensive assessment of nonspecific somatic symptoms, pain, and disability in children with HCTD to allow tailored treatment.

Assessing the Reliability of the OMERACT Juvenile Idiopathic Arthritis Magnetic Resonance Scoring System for Temporomandibular Joints (JAMRIS-TMJ)

Contrast-enhanced magnetic resonance imaging (MRI) remains the most comprehensive modality to assess juvenile idiopathic arthritis (JIA)-related inflammation and osteochondral damage in the temporomandibular joints (TMJ). This study tested the reliability of a new JIA MRI scoring system for TMJ (JAMRIS-TMJ) and the impact of variations in calibration and reader specialty. Thirty-one MRI exams of bilateral TMJs were scored independently using the JAMRIS-TMJ by 20 readers consisting of radiologists and non-radiologist clinicians in three reading groups, with or without a calibrating atlas and/or tutorial. The inter-reader reliability in the multidisciplinary cohort assessed by the generalizability coefficient was 0.61–0.67 for the inflammatory and 0.66–0.74 for the damage domain. The atlas and tutorial did not improve agreement within radiologists, but improved the agreement between radiologist and non-radiologist groups. Agreements between different calibration levels were 0.02 to 0.08 lower by the generalizability coefficient compared to agreement within calibration levels; agreement between specialty groups was 0.04 to 0.10 lower than within specialty groups. Averaging two radiologists raised the reliability above 0.8 for both domains. Therefore, the reliability of JAMRIS-TMJ was moderate-to-good depending on the presence of specialty and calibration differences. The atlas and tutorial are necessary to improve reliability when the reader cohort consists of multiple specialties.

Improving domain definition and outcome instrument selection: Lessons learned for OMERACT from imaging

OBJECTIVES

Imaging is one of the most rapidly evolving fields in medicine. Unfortunately, many imaging technologies have been applied as measurement instruments without rigorous evaluation of the evidence supporting their truth, discriminatory capability and feasibility for that context of use. The Outcome Measures in Rheumatology (OMERACT) Filter 2.1 Instrument Selection Algorithm (OFISA) is used to evaluate such evidence for use of an instrument in a research setting. The objectives of this work are to: [1] define and describe the key conceptual aspects that are essential for the evaluation of imaging as an outcome measurement instrument and [2] describe how these aspects can be assessed through OFISA.

METHODS

Experts in imaging and/or methodology met to formalize concepts and define key steps. These concepts were discussed with a team of patient research partners with interest in imaging to refine technical and methodological aspects into comprehensible information. A workshop was held at OMERACT2020 and feedback was incorporated into existing OMERACT process for domain and instrument selection.

RESULTS

Three key lessons were identified: (1) a clear definition of the domain we want to measure is a necessary prerequisite to the selection of a good instrument, (2) the sources of variability that can directly influence the instrument should be clearly identified, (3) incorporating these first two lessons into OFISA improves the quality of every instrument selection process.

CONCLUSIONS

The incorporation of these lessons in the updated OMERACT Filter (now 2.2) will improve the quality of the selection process for all types of outcome measurement instruments.

Discrete Choice Experiment on a Magnetic Resonance Imaging Scoring System for Temporomandibular Joints in Juvenile Idiopathic Arthritis

OBJECTIVE

To determine the relative importance weights of items and grades of a newly developed additive outcome measure called the juvenile idiopathic arthritis (JIA) magnetic resonance imaging (MRI) scoring system for temporomandibular joints (TMJ, JAMRIS-TMJ).

METHODS

An adaptive partial-profile discrete choice experiment (DCE) survey using the 1000Minds platform was independently completed by members of an expert group consisting of radiologists and non-radiologist clinicians to determine the group-averaged relative weights for JAMRIS-TMJ. Subsequently, an image-based vignette ranking exercise was done, during which experts individually rank-ordered 14 patient vignettes for disease severity while blinded to the weights and unrestricted to JAMRIS-TMJ assessment criteria. Validity of the weighted JAMRIS-TMJ was tested by comparing the consensus-graded, DCE-weighted JAMRIS-TMJ score of the vignettes with their unrestricted image-based ranks provided by the experts.

RESULTS

Nineteen experts completed the DCE survey and 21 completed the vignette ranking exercise. Synovial thickening and joint enhancement showed higher weights per raw score compared to bone marrow items and effusion in the inflammatory domain, while erosions and condylar flattening showed non-linear and higher weights compared to disk abnormalities in the damage domain. The weighted JAMRIS-TMJ score of the vignettes correlated highly with the ranks from the unrestricted comparison method, with median Spearman's rho of 0.92 (intra-quartile range: 0.87-0.95) for the inflammation and 0.93 (0.90-0.94) for the damage domain.

CONCLUSIONS

A DCE survey was used to quantify the importance weights of the items and grades of the JAMRIS-TMJ. The weighted score showed high convergent validity with an unrestricted, holistic vignette ranking method.

Toward Developing a Semiquantitative Whole Body-MRI Scoring for Juvenile Idiopathic Arthritis: Critical Appraisal of the State of the Art, Challenges, and Opportunities

With powerful new therapies available for management of juvenile idiopathic arthritis (JIA), early diagnosis leading to appropriate treatment may prevent long-term structural joint damage. Although magnetic resonance imaging (MRI) is typically used to assess individual body parts, indications for whole body (WB) MRI are increasing. Its utility as a diagnostic and monitoring tool has already been widely investigated in adult rheumatology patients, but less so in pediatric rheumatologic patients. This paper is a comprehensive review of scoring systems and a proposal for the conceptual development of a WB-MRI scoring system for the evaluation of JIA. In this review we identify, summarize, and critically appraise the available literature on the use of WB-MRI in inflammatory arthritis, addressing relevant considerations on components of a classification system that can lead to the development of a future pediatric WB-MRI scoring system for use in children with JIA. We also discuss advantages and challenges of developing such a WB-MRI scoring system for assessment of JIA and outline next steps toward the conceptual development of this scoring system.

Effect of the Inclusion of the Metacarpophalangeal Joints on the Wrist Magnetic Resonance Imaging Scoring System in Juvenile Idiopathic Arthritis

Objective.

To extend the magnetic resonance imaging (MRI) score for assessment of wrist synovitis in juvenile idiopathic arthritis (JIA) by inclusion of the metacarpophalangeal (MCP) joints, and to compare the metric properties of the original and the extended score.

Methods.

Wrist MRI of 70 patients with JIA were scored by 3 independent readers according to (1) the wrist component of the rheumatoid arthritis MRI synovitis score (comprising distal radioulnar, radiocarpal, and combined midcarpal and carpometacarpal joints); and (2) an extended score including the MCP joints. Thirty-eight patients had a 1-year MRI followup. The concordance between the readers [intraclass correlation coefficient (ICC), 95% limits of agreement (LOA), and weighted Cohen’s κ], correlations with clinical variables (Spearman’s ϱ), and the sensitivity to change [standardized response mean (SRM)] were calculated for both scores.

Results.

The interreader agreement was moderate for the original score (ICC 0.77; 95% CI 0.68–0.84) and good for the extended score (ICC 0.86; 95% CI 0.80–0.91). Using 95% LOA, the aggregate score variability was less favorable with relatively wide LOA. Weighted Cohen’s κ of the individual joints indicated good agreement for the original score and good to excellent agreement for the extended score. Correlations with clinical variables reflecting disease activity improved for the extended score and its SRM was higher compared to that of the original score.

Conclusion.

The extended score showed better reliability, construct validity, and sensitivity to change than the original. Inclusion of the MCP joints should be considered for a more accurate assessment of disease activity and treatment efficacy in JIA.

Toward Establishing a Standardized Magnetic Resonance Imaging Scoring System for Temporomandibular Joints in Juvenile Idiopathic Arthritis

OBJECTIVE

The temporomandibular joints (TMJs) are frequently affected in children with juvenile idiopathic arthritis (JIA). Early detection is challenging, as major variation is present in scoring TMJ pathology on magnetic resonance imaging (MRI). Consensus-driven development and validation of an MRI scoring system for TMJs has important clinical utility in timely improvement of diagnosis and serving as an outcome measure. We report on a multi-institutional collaboration toward developing a TMJ MRI scoring system for JIA.

METHODS

Seven readers independently assessed MRI scans from 21 patients (42 TMJs, from patients ages 6-16 years) using 3 existing MRI scoring systems from American, German, and Swiss institutions. Reliability scores, scoring system definitions, and items were discussed among 10 JIA experts through 2 rounds of Delphi surveys, nominal group voting, and subsequent consensus meetings to create a novel TMJ MRI scoring system.

RESULTS

Average-measure absolute agreement intraclass correlation coefficients (avICCs) for the total scores of all 3 scoring systems were highly reliable at 0.96 each. Osteochondral items showed higher reliability than inflammatory items. An additive system was deemed preferable for assessing minor joint changes over time. Eight items were considered sufficiently reliable and/or important for integration into the consensus scoring system: bone marrow edema and enhancement (avICC 0.57-0.61, smallest detectable difference [SDD] ± 45-63% prior to redefining), condylar flattening (avICC 0.95-0.96, SDD ± 23-28%), effusions (avICC 0.85-0.88, SDD ± 25-26%), erosions (avICC 0.94, SDD ± 20%), synovial enhancement and thickening (previously combined, avICC 0.90-0.91, SDD ± 33%), and disk abnormalities (avICC 0.90, SDD ± 19%).

CONCLUSION

A novel TMJ MRI scoring system was developed by consensus. Further iterative refinements and reliability testing are warranted in upcoming studies.

Quality of life and behavioral functioning in Dutch children with a history of Kawasaki disease

OBJECTIVE

The authors evaluated health-related quality of life (HRQOL) and behavioral functioning in patients with a history of Kawasaki disease (KD).

STUDY DESIGN

A cross-sectional study was conducted at a tertiary referral center for KD follow-up in 280 patients (mean age 8.6 years, 60.0% male). Patients were eligible when they were aged 0-18 years and had a history of KD. HRQOL was assessed using the TNO-AZL Preschool Children Quality of Life questionnaire for children 0-5 years old and the Pediatric Inventory of Quality of Life Core Scales 4.0 for those 6-18 years old. Behavioral functioning was evaluated using the Strength and Difficulties Questionnaire (8-16 years proxy report and 11-16 years self-report). KD results were compared with Dutch norm data, and patients with and without coronary artery aneurysms were compared.

RESULTS

HRQOL was significantly worse for male patients aged 0-5 years on 4 of the 12 TNO-AZL Preschool Children Quality of Life questionnaire scales and for female patients on the motor functioning scale. At an older age, the HRQOL of patients was comparable with the norm population. Coronary artery status did not influence HRQOL. Parents reported more behavioral problems on the hyperactivity and emotional subscale in patients compared with the norm population.

CONCLUSIONS

Although at an older age the HRQOL of patients with KD is comparable with the Dutch norm, HRQOL seems to be particularly impaired at younger age. Parents reported more hyperactivity and emotional problems in patients with KD.