Accuracy and Validity of Computer Adaptive Testing for Outcome Assessment in Patients Undergoing Total Hip Arthroplasty

Psychometric properties of computerized adaptive testing for chronic obstructive pulmonary disease patient-reported outcome measurement

BACKGROUND

Computerized adaptive testing (CAT) is an effective way to reduce time, repetitious redundancy, and respond burden, and has been used to measure outcomes in many diseases. This study aimed to develop and validate a comprehensive disease-specific CAT for chronic obstructive pulmonary disease (COPD) patient-reported outcome measurement.

METHODS

The discrimination and difficulty of the items from the modified patient-reported outcome scale for COPD (mCOPD-PRO) were analyzed using item response theory. Then the initial item, item selection method, ability estimation method, and stopping criteria were further set based on Concerto platform to form the CAT. Finally, the reliability and validity were validated.

RESULTS

The item discrimination ranged from 1.05 to 2.71, and the item difficulty ranged from - 3.08 to 3.65. The measurement reliability of the CAT ranged from 0.910 to 0.922 using random method, while that ranged from 0.910 to 0.924 using maximum Fisher information (MFI) method. The content validity was good. The correlation coefficient between theta of the CAT and COPD assessment test and modified Medical Research Council dyspnea scale scores using random method was 0.628 and 0.540 (P < 0.001; P < 0.001) respectively, while that using MFI method was 0.347 and 0.328 (P = 0.007; P = 0.010) respectively. About 11 items (reducing by 59.3%) on average were tested using random method, while about seven items (reducing by 74.1%) on average using MFI method. The correlation coefficient between theta of the CAT and mCOPD-PRO total scores using random method was 0.919 (P < 0.001), while that using MFI method was 0.760 (P < 0.001).

CONCLUSIONS

The comprehensive disease-specific CAT for COPD patient-reported outcome measurement is well developed with good psychometric properties, which can provide an efficient, accurate, and user-friendly measurement for patient-reported outcome of COPD.

Research Trends of Patient-Reported Outcome Measures in Orthopedic Medical Practices: A Bibliometric and Visualized Study

Background and Objectives: Patient-reported outcome measures (PROMs), also known as self-report measures, are critical tools for evaluating health outcomes by gathering information directly from patients without external interpretation. There has been a growing trend in the number of publications focusing on PROMs in orthopedic-related research. This study aims to identify the most valuable publications, influential journals, leading researchers, and core countries in this field using bibliometric analysis, providing researchers with an understanding of the current state and future trends of PROMs in orthopedic research. Materials and Methods: All PROMs in orthopedic-related publications from 1991 to 2022 were obtained from the WoSCC database. R software (version 4.2.2), VOSviewer (version 1.6.17), and Microsoft Excel (version 2303) were used for the bibliometric and visual analysis. Results: A total of 2273 publication records were found from 1991 to 2022. The results indicated that the United States (US) has made significant contributions to orthopedic-related PROMs. The majority of active research institutions are located in the US. J ORTHOP RES has published the most articles. J BONE JOINT SURG AM has the highest total citations. Conclusions: Our study provides a valuable reference for further exploration of the application of PROMs in orthopedics. PROMs have emerged as an increasingly popular area of research within the field of orthopedics, both in clinical practice and academic research. We conducted a bibliometric analysis in terms of journals, authors, countries, and institutions in this field. Additionally, we analyzed the potentialities and advantages of using PROMs in orthopedic research. There is an increasing trend towards using network-based or short message service (SMS)-based electronic patient-reported outcome measures (ePROMs) in orthopedic medical practices. It is anticipated that the role of PROMs in psychological and mental health research and telemedicine will continue to grow in importance.

Efficiency and Accuracy of Computerized Adaptive Testing for the Oswestry Disability Index and Neck Disability Index

This study aimed to determine the efficiency and accuracy of computerized adaptive testing (CAT) models of the Oswestry Disability Index (ODI) and Neck Disability Index (NDI).

Methods

The study involved simulation using retrospectively collected real-world data. Previously developed CAT models of the ODI and NDI were applied to the responses from 52,551 and 18,196 patients with spinal conditions, respectively. Efficiency was evaluated by the reduction in the number of questions administered. Accuracy was evaluated by comparing means and standard deviations, calculating Pearson r and intraclass correlation coefficient (ICC) values, plotting the frequency distributions of CAT and full questionnaire scores, plotting the frequency distributions of differences between paired scores, and Bland-Altman plotting. Score changes, calculated as the postoperative ODI or NDI scores minus the preoperative scores, were compared between the CAT and full versions in patients for whom both preoperative and postoperative ODI or NDI questionnaires were available.

Results

CAT models of the ODI and NDI required an average of 4.47 and 4.03 fewer questions per patient, respectively. The mean CAT ODI score was 0.7 point lower than the full ODI score (35.4 ± 19.0 versus 36.1 ± 19.3), and the mean CAT NDI score was 1.0 point lower than the full NDI score (34.7 ± 19.3 versus 33.8 ± 18.5). The Pearson r was 0.97 for both the ODI and NDI, and the ICC was 0.97 for both. The frequency distributions of the CAT and full scores showed marked overlap for the ODI and NDI. Differences between paired scores were less than the minimum clinically important difference in 98.9% of cases for the ODI and 98.5% for the NDI. Bland-Altman plots showed no proportional bias. The ODI and NDI score changes could be calculated in a subgroup of 6,044 and 4,775 patients, respectively; the distributions of the ODI and NDI score changes were near identical between the CAT and full versions.

Conclusions

CAT models were able to reduce the question burden of the ODI and NDI. Scores obtained from the CAT models were faithful to those from the full questionnaires, both on the population level and on the individual patient level.

Level of Evidence

Prognostic Level III. See Instructions for Authors for a complete description of levels of evidence.

The Application of Computerized Adaptive Testing to the International Knee Documentation Committee Subjective Knee Evaluation Form

BACKGROUND

Patient-reported outcome measures (PROMs) are commonly used to monitor functional outcomes for clinical and research purposes; unfortunately, many PROMs include redundant, burdensome questions for patients. The use of predictive models to implement computerized adaptive testing (CAT) offer a potential solution to reduce question burden in outcomes research.

PURPOSE

To validate the usage of an appropriate CAT system to improve the efficiency of the International Knee Documentation Committee (IKDC) Subjective Knee Form.

STUDY DESIGN

Cohort study (Diagnosis); Level of evidence, 2.

METHODS

Validation was based on electronically collected patient responses from 2 separate orthopaedic sports medicine clinics. Diagnoses included, but were not limited to, meniscal lesions, ligamentous injuries, and chondral defects. The CAT system was previously developed through analysis of an electronic knee PROM database that did not contain any of these cases.

RESULTS

A total of 2173 patient responses (1229 patients) were collected. The CAT model was able to reduce the question burden by a mean of 9.33 questions (45.1%). Higher CAT-predicted scores correlated strongly with higher actual scores (r = 0.99; intraclass correlation coefficient = 0.99). The mean difference between the CAT-predicted score and the actual PROM score was 0.48 of a point on a scale of 0 to 100.

CONCLUSION

The use of CAT systems, in conjunction with electronic PROMs, can accurately predict outcome scores for IKDC PROMs, while dramatically decreasing the number of questionnaire items needed for any given patient. By decreasing questionnaire burden, clinicians and researchers can potentially increase patient participation and follow-up in both clinical assessments and research trials.