Updating the OMERACT filter: discrimination and feasibility

[Psychometric properties of performance outcome tools to assess sensitivity in subjects with stroke sequela: A systematic review]

The objective was to perform a systematic review of those performance outcomes (HRD) tools that assess sensitivity in post-stroke subjects, as well as to know which of all is the most reliable and viable. The design was a systematic review. The following were included: HRD, developed in post-stroke subjects, of any etiology and period of evolution; 18 years of age or older and testing for sensitivity; results on psychometric properties. Excluded: self-reported results, the report of cross-cultural adaptation, designed through computerized means. 19 HRD were acquired. The psychometric properties analyzed were reliability, internal consistency, measurement error, construct validity, content validity, and sensitivity to change, the first being the most studied. Psychometric properties were identified, assessed, compared, and summarized. Of all of them, we recommend the use of the modified Erasmus Nottingham Sensory Assessment, as it presents the best degree of confidence in the evidence.

Computer‐Assisted Image Analysis in Assessment of Peripheral Joint MRI in Inflammatory Arthritis: A Systematic Review and Meta‐analysis

Objective

To summarize the feasibility of computer‐assisted quantification of joint pathologies on magnetic resonance imaging (MRI) in patients with inflammatory arthritis by evaluating the published data on reliability, validity, and feasibility.

Methods

A systematic literature search was performed for original articles published from January 1, 1985, to January 1, 2021. We selected studies in which patients with inflammatory arthritis were enrolled, and arthritis‐related structural damage/synovitis in peripheral joints was assessed on non‐contrast‐enhanced, contrast‐enhanced (CE), or dynamic CE (DCE)‐MRI using (semi)automated methods. Data were pooled using random‐effects model.

Results

Twenty‐eight studies consisting of 1342 MRIs were included (mean age, 54.8 years; 66.7% female; duration of arthritis, 3.6 years). Among clinical/laboratory factors, synovial membrane volume (SV) was moderately correlated with erthrocyte sedimentation rate (ESR) level (P < 0.01). Pooled analysis showed an overall excellent intra‐ and inter‐reader reliability for computer‐aided quantification of bone erosion volume (BEV; r = 0.97 [95% CI: 0.92‐0.99], 0.93 [0.87‐0.97]), SV (r = 0.98 [95% CI: 0.90‐0.99], 0.86 [0.78‐0.91]), and DCE‐MRI perfusion parameters (r = 0.96‐0.99). Meta‐regression showed that computer‐aided and manual methods provide comparable reliability (P > 0.05). Computer‐aided measurement of BEV (r = 0.92), SV (r = 0.82), and DCE‐MRI biomarkers (r = 0.72 N‐total; r = 0.74 N‐plateau; r = 0.64 N‐washout) were significantly correlated with the Rheumatoid Arthritis Magnetic Resonance Imaging Score (RAMRIS; P < 0.01), allowing for earlier assessment of drug efficacy. On average, (semi)automated analysis of BEV/SV took 17 minutes (vs. 9 minutes for the RAMRIS) and DCE‐MRI took 4 minutes (vs. 33 minutes for manual assessment).

Conclusion

Computer‐aided image quantification technologies demonstrate excellent reliability and validity when used to quantify MRI pathologies of peripheral joints in patients with inflammatory arthritis. Computer‐aided evaluation of inflammatory arthritis is an emerging field and should be considered as a viable complement to conventional observer‐based scoring methods for clinical trials application.

Reliability of Magnetic Resonance Imaging (MRI) Scoring of the Metatarsophalangeal Joints of the Foot according to the Rheumatoid Arthritis MRI Score

OBJECTIVE

The Rheumatoid Arthritis Magnetic Resonance Imaging Score (RAMRIS) is validated for hand MRI. Its reliability applied to metatarsophalangeal (MTP 1-5) joints is unknown and was studied in early arthritis and clinically suspect arthralgia.

METHODS

Patients underwent 1.5 Tesla MRI of MTP, metacarpophalangeal (MCP 2-5), and wrist joints. Two paired readers scored bone marrow edema (BME), synovitis, tenosynovitis, and erosions. Interreader reliability was assessed of 441 consecutive early arthritis patients at baseline, 215 by 2 readers, and the remaining 226 by 2 different readers. Two readers scored baseline MRI of 82 consecutive patients with clinically suspect arthralgia, and 40 randomly selected patients by 9 readers. Intrareader reliability was determined on a random set of 15 early arthritis patients, scored twice by 2 readers. For change scores, 30 early arthritis patients with baseline and 1-year followup MRI were scored by 2 readers. Intraclass correlation coefficients (ICC), Bland-Altman (BA) plots, and smallest detectable change (SDC) were determined. MRI data of MTP joints were compared to wrist and MCP joints.

RESULTS

Interreader ICC and mean scores in early arthritis were BME ICC 0.91-0.92 (mean 1.5 ± SD 2.6), synovitis 0.90-0.92 (1.3 ± 1.7), tenosynovitis 0.80-0.85 (1.1 ± 1.8), and erosions 0.88-0.89 (0.7 ± 1.0). In patients with clinically suspect arthralgia, ICC were comparable. Intrareader ICC for inflammatory MRI features were 0.84-0.98, for erosions 0.71 (reader 1), and 0.92 (reader 2). Change score ICC were ≥ 0.90, except erosions (0.77). SDC were ≤ 1.0. BA plots showed no systematic bias. Reliability scores of MTP joints were similar to MCP and wrist joints.

CONCLUSION

Status and change MRI scores of BME, synovitis, tenosynovitis, and erosions of MTP joints can be assessed reliably by RAMRIS.

PROMIS Physical Function Short Forms Display Item- and Scale-Level Characteristics at Least as Good as the Roland Morris Disability Questionnaire in Patients With Chronic Low Back Pain

OBJECTIVE

To compare dimensionality, item-level characteristics, scale-level reliability, and construct validity of PROMIS Physical Function short forms (PROMIS-PF) and 24-item Roland Morris Disability Questionnaire (RMDQ-24) in patients with chronic low back pain (LBP).

DESIGN

Cross-sectional study.

SETTING

Secondary care center for rehabilitation and rheumatology.

PARTICIPANTS

Patients with nonspecific LBP ≥3 months (N=768). Mean age was 49±13 years, 77% were female, and 54% displayed pain for more than 5 years.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Dutch versions of the 4-, 6-, 8-, 10-, and 20-item PROMIS-PF and of the RMDQ-24.

RESULTS

PROMIS-PF-6, PROMIS-PF-8, and RMDQ-24 exhibited sufficient unidimensionality (confirmatory factor analysis: comparative fit index>0.950, Tucker-Lewis index>0.950, root means square error of approximation<0.060), whereas the other instruments did not. All instruments were free of local dependence except PROMIS-PF-20 with 4 item pairs with clear residual correlations. Mokken scale analysis found 1 nonmonotone item for PROMIS-PF-20 and 8 for RMDQ-24 (ie, the probability of endorsing these items was not increasing with increasing level on the underlying construct). PROMIS-PF-20 displayed 2 misfitting items (S-χ²P value>.001). Two-parameter item response theory models found 2 items with low discrimination for RMDQ-24. All other instruments had adequate fit statistics and item parameters. PROMIS-PF-20 displayed the best scale-level reliability. Construct validity was sufficient for all instruments as all hypotheses on expected correlations with other instruments and differences between relevant subgroups were met.

CONCLUSIONS

PROMIS-PF-6, PROMIS-PF-8, and RMDQ-24 exhibited better unidimensionality, whereas PROMIS-PF-4, PROMIS-PF-6, PROMIS-PF-8, and PROMIS-PF-10 showed superior item-level characteristics. PROMIS-PF-20 was the instrument with the best scale-level reliability. This study warrants assessment of other measurement properties of PROMIS-PF short forms in comparison with disease-specific physical functioning instruments in LBP.

Utilize lung ultrasound B-lines and KL-6 to monitor anti-MDA-5 antibody-positive clinically amyopathic dermatomyositis-associated interstitial lung disease: a case report and literature review

In the past decade, lung ultrasound (LUS) B-lines and serum Krebs von den Lungen-6 (KL-6) antigen have been recognized as biomarkers of the connective tissue disease-associated interstitial lung diseases (CTD-ILDs). Robust data have demonstrated that B-lines total numbers and KL-6 levels are correlated with high-resolution computed tomography findings, pulmonary function test, and some clinical parameters in CTD-ILDs. However, limited data are available regarding the use of these two biomarkers to follow CTD-ILDs. Herein, we report a case with anti-melanoma differentiation-associated gene 5 antibody-positive clinically amyopathic dermatomyositis-associated ILD, successfully treated with high-dose methylprednisolone, cyclophosphamide, intravenous immunoglobulin, pirfenidone, and followed using lung ultrasound and KL-6.

Validity of the rheumatoid arthritis MRI score applied to the forefeet using the OMERACT filter: a systematic literature review

Objective

MRI depicts inflammation and structural damage in rheumatoid arthritis (RA). The validity of MRI-scoring of wrist-joints and metacarpophalangeal-joints according to the RA MRI score(RAMRIS) has been demonstrated. The Outcomes in Rheumatology Clinical Trials (OMERACT) RAMRIS Working Group recently called for validation of the RAMRIS of the metatarsophalangeal (MTP)-joints. Therefore, a systematic literature review was performed to test if the RAMRIS applied to the MTP-joints meets the OMERACT Filter of Truth, Discrimination and Feasibility.

Methods

Medical literature databases up to January 2018 were systematically reviewed for studies reporting on RAMRIS applied to MRI of the MTP-joints in RA. To be included, an article had to contain at least one MRI-feature (synovitis, bone marrow oedema (BME), tenosynovitis, erosion, joint space narrowing (JSN)) and one item from the OMERACT Filter: Truth (face, content and construct validity), Discrimination (test-retest reliability, ability to discriminate in trials, longitudinal construct validity and thresholds of meaning) and Feasibility.

Results

Of the 749 retrieved studies, 13 were included, of which 9 provided data on construct validity, 4 on discrimination (3 on reliability, 2 on longitudinal construct validity and 1 on ability to discriminate in trials) and none on feasibility. Construct validity was suggested for BME and erosions, but lacking for synovitis, tenosynovitis and JSN. Data for discrimination remain to be developed for all outcomes.

Conclusion

According to the OMERACT Filter, the validity of the RAMRIS of the forefeet is insufficient in different aspects. A research agenda was determined.

Report of the GRAPPA-OMERACT Psoriatic Arthritis Working Group from the GRAPPA 2015 Annual Meeting

The GRAPPA-OMERACT psoriatic arthritis (PsA) working group is in the process of updating the PsA core domain set to improve and standardize the measurement of PsA outcomes. Work streams comprise literature reviews of domains and outcome measurement instruments, an international qualitative research project with PsA patients to generate domains important to patients, outcome measurement instrument assessment, conduct of domain consensus panels with patients and physicians, and evidence-based selection of instruments. Patient research partners are involved in each of the projects. The working group will present findings and seek endorsement for the new PsA core domain set, outcome measurement set, and research agenda at the OMERACT meeting in May 2016.

Defining Skin Ulcers in Systemic Sclerosis: Systematic Literature Review and Proposed World Scleroderma Foundation (WSF) definition

Purpose

There is a lack of a valid, definition for skin ulcers in SSc to be used in clinical trials. Our aim was to develop a consensus definition for SSc-skin ulcers based on the results of a systematic literature review (SLR) for skin ulcer definitions and expert opinion; and to evaluate its face validity, reliability and feasibility.

Methods

SLR for skin ulcer definitions was conducted using PubMed, Web of Science, and Cochrane library for articles published from inception to January 1^st, 2016. SSc experts were to discuss the definitions' categories and vote for the relevant terms. Reliability of the definition were tested in a second expert meeting, seven SSc experts evaluated 7 SSc pts with skin lesions twice. Face validity and feasibility evaluated by sending out case report forms(CRFs) to 4 SSc experts, they were asked to use the definition in 5 pts each.

Results

A total of 3464 abstracts and titles were screened, and 446 articles were fully evaluated. Of these, 66 met eligibility criteria and skin ulcer definitions were extracted. SSc experts discussed, refined and voted on the consensus definition using nominal process. Kappa for inter-, intra-rater rater agreement was 0.51, 0.90 respectively. The mean time to decide if the lesion is an ulcer was 7.4 sec. All investigators endorsed the face validity of the new definition in the CRFs.

Conclusion

Using a SLR and a nominal technique, we developed a preliminary consensus-based definition of SSc-skin ulcers. Face validity, feasibility and reliability were demonstrated for the developed definition.

Core outcome sets for research and clinical practice

BACKGROUND

This masterclass introduces the topic of core outcome sets, describing rationale and methods for developing them, and providing some examples that are relevant for clinical research and practice.

METHOD

A core outcome set is a minimum consensus-based set of outcomes that should be measured and reported in all clinical trials for a specific health condition and/or intervention. Issues surrounding outcome assessment, such as selective reporting and inconsistency across studies, can be addressed by the development of a core set. As suggested by key initiatives in this field (i.e. OMERACT and COMET), the development requires achieving consensus on: (1) core outcome domains and (2) core outcome measurement instruments. Different methods can be used to reach consensus, including: literature systematic reviews to inform the process, qualitative research with clinicians and patients, group discussions (e.g. nominal group technique), and structured surveys (e.g. Delphi technique). Various stakeholders should be involved in the process, with particular attention to patients.

RESULTS AND CONCLUSIONS

Several COSs have been developed for musculoskeletal conditions including a longstanding one for low back pain, IMMPACT recommendations on outcomes for chronic pain, and OMERACT COSs for hip, knee and hand osteoarthritis. There is a lack of COSs for neurological, geriatric, cardio-respiratory and pediatric conditions, therefore, future research could determine the value of developing COSs for these conditions.

A Call for Evidence-based Decision Making When Selecting Outcome Measurement Instruments for Summary of Findings Tables in Systematic Reviews: Results from an OMERACT Working Group

OBJECTIVE

Systematic reviews often struggle with how to combine information when more than 1 instrument is used across studies being synthesized. Different techniques have been suggested based on frequency of use in the literature, or on consensus. We explore an approach blending 2 initiatives: OMERACT (Outcome Measurement in Rheumatology) and COSMIN (Consensus On Selection of Measurement Instruments), and investigate the effects of an evidence-based measurement approach on selection of outcomes.

METHODS

Readings were circulated to attendees registered for a preconference workshop on pain measurement. Three instruments were considered and exercises conducted to engage people in the content and measurement performance of these tools. Consensus was sought that an evidence-based approach could be created for selection of instruments for summary of findings (SoF) tables.

RESULTS

The blending of COSMIN and OMERACT approaches led to an evidence-based approach that depended both on a clear definition of target concept and a review of measurement performance of the instrument. Participants emphasized that conceptual clarity and practical considerations should come before measurement property results.

CONCLUSION

Evidence-based approaches can be adopted for selection of instruments for SoF tables. A research agenda was formulated.

Outcome Domains and Measures in Total Joint Replacement Clinical Trials: Can We Harmonize Them? An OMERACT Collaborative Initiative

Objective.

To develop a plan for harmonizing outcomes for people undergoing total joint replacement (TJR), to achieve consensus regarding TJR outcome research.

Methods.

The TJR working group met during the 2014 Outcome Measures in Rheumatology (OMERACT) 12 meeting in Budapest, Hungary. Multiple conference calls preceded the face-to-face meeting. Brief presentations were made during a 1.5-h meeting, which included an overview of published systematic reviews of TJR trials and the results of a recent systematic review of TJR clinical trial outcome domains and measures. This was followed by discussion of potential core set areas/domains for TJR clinical trials (as per OMERACT Filter 2.0) as well as the challenges associated with the measurement of these domains.

Results.

Working group participants discussed which TJR clinical trial outcome domains/areas map to the inner versus outer core for core domain set. Several challenges were identified with TJR outcomes including how to best measure function after TJR, elucidating the source of the pre- and post-TJR joint pain being measured, joint-specific versus generic quality of life instruments and the importance of patient satisfaction and revision surgery as outcomes. A preliminary core domain set for TJR clinical trials was proposed and included pain, function, patient satisfaction, revision, adverse events, and death. This core domain set will be further vetted with a broader audience.

Conclusion.

An international effort with active collaboration with the orthopedic community to standardize key outcome domains and measures is under way with the TJR working group. This effort will be further developed with new collaborations.

Updating the OMERACT filter at OMERACT 11

The Outcome Measures in Rheumatology (OMERACT) community strives to develop core outcome sets for rheumatologic conditions to specify, for each condition, the minimum set of outcomes necessary to provide consistent estimates of the benefits of an intervention. The original and successful OMERACT filter of "truth, discrimination, and feasibility" requires development and updating because of application to a widening range of conditions by an expanding group, particularly patients. It should more explicitly identify the relevant core outcomes that might be universal to all randomized controlled trials within rheumatology. Working from first principles, comparing proposals against actual procedures adopted by OMERACT working groups, and seeking a broad consensus over several major sessions at the OMERACT 11 meeting, a new version has emerged, OMERACT Filter 2.0, which will form the central theme of the intended OMERACT handbook and offers an approach to core outcome set development in many areas of healthcare.

Updating the OMERACT filter: implications of filter 2.0 to select outcome instruments through assessment of "truth": content, face, and construct validity

OBJECTIVE

The Outcome Measures in Rheumatology (OMERACT) Filter provides guidelines for the development and validation of outcome measures for use in clinical research. The "Truth" section of the OMERACT Filter requires that criteria be met to demonstrate that the outcome instrument meets the criteria for content, face, and construct validity.

METHODS

Discussion groups critically reviewed a variety of ways in which case studies of current OMERACT Working Groups complied with the Truth component of the Filter and what issues remained to be resolved.

RESULTS

The case studies showed that there is broad agreement on criteria for meeting the Truth criteria through demonstration of content, face, and construct validity; however, several issues were identified that the Filter Working Group will need to address.

CONCLUSION

These issues will require resolution to reach consensus on how Truth will be assessed for the proposed Filter 2.0 framework, for instruments to be endorsed by OMERACT.

How to choose core outcome measurement sets for clinical trials: OMERACT 11 approves filter 2.0

OBJECTIVE

The Outcome Measures in Rheumatology (OMERACT) initiative works to develop core sets of outcome measures for trials and observational studies in rheumatology. At the OMERACT 11 meeting, substantial time was devoted to discussing a conceptual framework and a proposal for a more explicit working process to develop what we now propose to term core outcome measurement sets, collectively termed "OMERACT Filter 2.0."

METHODS

Preconference work included a literature review, and discussion of preliminary proposals through face-to-face discussions and Internet-based surveys with people within and outside rheumatology. At the conference, 5 interactive sessions comprising plenary and small-group discussions reflected on the proposals from the viewpoint of previous and ongoing OMERACT work. These considerations were brought together in a final OMERACT presentation seeking consensus agreement for the Filter 2.0 framework.

RESULTS

After debate, clarification, and agreed alterations, the final proposal suggested all core sets should contain at least 1 measurement instrument from 3 Core Areas: Death, Life Impact, and Pathophysiological Manifestations, and preferably 1 from the area Resource Use. The process of core set development for a health condition starts by selecting core domains within the areas ("core domain set"). This requires literature searches, involvement (especially of patients), and at least 1 consensus process. Next, developers select at least 1 applicable measurement instrument for each core domain. Applicability refers to the original OMERACT Filter and means that the instrument must be truthful (face, content, and construct validity), discriminative (between situations of interest) and feasible (understandable and with acceptable time and monetary costs). Depending on the quality of the instruments, participants formulate either a preliminary or a final "core outcome measurement set." At final vote, 96% of participants agreed "The proposed overall framework for Filter 2.0 is a suitable basis on which to elaborate a Filter 2.0 Handbook."

CONCLUSION

Within OMERACT, Filter 2.0 has made established working processes more explicit and includes a broadly endorsed conceptual framework for core outcome measurement set development.

Toward a generalized framework of core measurement areas in clinical trials: a position paper for OMERACT 11

OBJECTIVE

The Outcome Measures in Rheumatology (OMERACT) international consensus initiative has successfully developed core sets of outcome measures for trials of many rheumatologic conditions, but its expanding scope called for clarification and updating of its underlying conceptual framework and working process. To develop a core set of what we propose to call outcome measurement instruments, consensus must be reached both on what to measure and how to measure. This article deals with the first part: a framework necessary to ensure comprehensiveness of the domains chosen for measurement. We formulated a conceptual framework of core measurement areas in clinical trials, for discussion at the OMERACT 11 conference.

METHODS

We formulated a framework and definitions of key concepts adapted from the literature, and followed an iterative consensus process (small group processes and an Internet-based survey) of those involved including patients, health professionals, and methodologists within and outside rheumatology.

RESULTS

The draft framework comprises 4 core "areas": death, life impact (all aspects of how a patient feels or functions), resource use (monetary and other costs of the health condition and interventions), and pathophysiologic manifestations (disease-specific clinical and psychological signs, biomarkers, and potential surrogate outcome measures necessary to assess specific effects). The survey responses (262 of 2293, response rate 11%) indicated broad agreement with the draft framework and the proposed definitions of key concepts, including understandability and feasibility. A total of 283 comments were processed.

CONCLUSION

In an iterative process, we have developed a generic framework for outcome measurement and working definitions of key concepts ready for discussion at the OMERACT 11 conference.

Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0

BACKGROUND

Lack of standardization of outcome measures limits the usefulness of clinical trial evidence to inform health care decisions. This can be addressed by agreeing on a minimum core set of outcome measures per health condition, containing measures relevant to patients and decision makers. Since 1992, the Outcome Measures in Rheumatology (OMERACT) consensus initiative has successfully developed core sets for many rheumatologic conditions, actively involving patients since 2002. Its expanding scope required an explicit formulation of its underlying conceptual framework and process.

METHODS

Literature searches and iterative consensus process (surveys and group meetings) of stakeholders including patients, health professionals, and methodologists within and outside rheumatology.

RESULTS

To comprehensively sample patient-centered and intervention-specific outcomes, a framework emerged that comprises three core "Areas," namely Death, Life Impact, and Pathophysiological Manifestations; and one strongly recommended Resource Use. Through literature review and consensus process, core set development for any specific health condition starts by identifying at least one core "Domain" within each of the Areas to formulate the "Core Domain Set." Next, at least one applicable measurement instrument for each core Domain is identified to formulate a "Core Outcome Measurement Set." Each instrument must prove to be truthful (valid), discriminative, and feasible. In 2012, 96% of the voting participants (n=125) at the OMERACT 11 consensus conference endorsed this model and process.

CONCLUSION

The OMERACT Filter 2.0 explicitly describes a comprehensive conceptual framework and a recommended process to develop core outcome measurement sets for rheumatology likely to be useful as a template in other areas of health care.