PROMIS and FAAM Minimal Clinically Important Differences in Foot and Ankle Orthopedics

What Are the Floor and Ceiling Effects of Patient-Reported Outcomes Measurement Information System Computer Adaptive Test Domains in Orthopaedic Patients? A Systematic Review

PURPOSE

To perform a systematic review to answer the following: (1) What are the floor and ceiling (F/C) effects of the Patient-Reported Outcomes Measurement Information System (PROMIS) computer adaptive test (CAT) domains of physical function (PF), upper extremity physical function (UE), pain interference (PI), and depression (D) in adult orthopaedic patients? (2) Do the PROMIS-PF and PROMIS-PI domains have differing F/C effects depending on use in upper extremity, lower extremity, spine, neck, and back, or trauma patients?.

METHODS

(Preferred Reporting Items for Systematic Review and Meta-Analyses guidelines were followed, the review was registered on PROSPERO, and the methodological index for non-randomized studies was used for this systematic review. Studies reporting the F/C effects of at least 1 of 4 PROMIS CAT domains in orthopaedic patient cohorts accessed through PubMed and Embase on October 30, 2018, were included. F/C effects for each study were reported within forest plots.

RESULTS

Forty-three studies were included. Generally, varying cohorts demonstrated no F/C effects for PROMIS-PF (0%-9.0%), variable ceiling effects for PROMIS-UE (lower in v2.0; 0%-28.2%), variable floor effects for PROMIS-PI (0%-19.0%), and significant floor effects for PROMIS-D (0.4%-23.4%).

CONCLUSIONS

The orthopaedic literature demonstrated generally favorable floor and ceiling effects for PROMIS CAT domains, with the exception of variable ceiling effects for PROMIS-UE (the newer version exhibits only minor effects), variable floor effects for PROMIS-PI, and significant floor effects for PROMIS-D. In addition, the F/C effects of PROMIS-PF did not vary based on patient population. Although the floor effects of PROMIS-PI did vary based on patient population, the variability does not appear to be based solely on anatomic location. The PROMIS-PF and PROMIS-UE v2.0 demonstrate consistently low floor and ceiling effects. However, the PROMIS-PI and PROMIS-D may need modification before widespread adoption for clinical and research purposes.

LEVEL OF EVIDENCE

III; systematic review of Level I-III studies.

Responsiveness of the PROMIS and its Concurrent Validity with Other Region- and Condition-specific PROMs in Patients Undergoing Carpal Tunnel Release

BACKGROUND

The Patient-reported Outcome Measurement Information System (PROMIS) continues to be an important universal patient-reported outcomes measure (PROM) in orthopaedic surgery. However, there is concern about the performance of the PROMIS as a general health questionnaire in hand surgery compared with the performance of region- and condition-specific PROMs such as the Michigan Hand Questionnaire (MHQ) and the Boston Carpal Tunnel Questionnaire (BCTQ), respectively. To ensure that PROMIS domains capture patient-reported outcomes to the same degree as region- and condition-specific PROMs do, comparing PROM performance is necessary.

QUESTIONS/PURPOSES

(1) Which PROMs demonstrate high responsiveness among patients undergoing carpal tunnel release (CTR)? (2) Which of the PROMIS domains (Physical Function [PF], Upper Extremity [UE], and Pain Interference [PI]) demonstrate concurrent validity with the HHQ and BCTQ domains?

METHODS

In this prospective study, between November 2014 and October 2016, patients with carpal tunnel syndrome visiting a single surgeon who elected to undergo CTR completed the BCTQ, MHQ, and PROMIS UE, PF, and PI domains at each visit. A total of 101 patients agreed to participate. Of these, 31 patients (31%) did not return for a followup visit at least 6 weeks after CTR and were excluded, leaving a final sample of 70 patients (69%). We compared the PROMIS against region- and condition-specific PROMs in terms of responsiveness and concurrent validity. Responsiveness was determined using Cohen's d or the effect-size index (ESI). The larger the absolute value of the ESI, the greater the effect size. Using the ESI allows surgeons to better quantify the impact of CTR, with a medium ESI (that is, 0.5) representing a visible clinical change to a careful observer. Concurrent validity was determined using Spearman's correlation coefficient with correlation strengths categorized as excellent (> 0.7), excellent-good (0.61-0.70), good (0.4-0.6), and poor (< 0.4). Significance was set a priori at p < 0.05.

RESULTS

Among PROMIS domains, the PI demonstrated the best responsiveness (ESI = 0.74; 95% CI, 0.39-1.08), followed by the UE (ESI = -0.66; 95% CI, -1.00 to -0.31). For the MHQ, the Satisfaction domain had the largest effect size (ESI = -1.48; 95% CI, -1.85 to -1.09), while for the BCTQ, the Symptom Severity domain had the best responsiveness (ESI = 1.54; 95% CI, 1.14-1.91). The PROMIS UE and PI domains demonstrated excellent-good to excellent correlations to the total MHQ and BCTQ-Functional Status scores (preoperative UE to MHQ: ρ = 0.68; PI to MHQ: ρ = 0.74; UE to BCTQ-Functional Status: ρ = 0.74; PI to BCTQ-Functional Status: ρ = 0.67; all p < 0.001), while the PROMIS PF demonstrated poor correlations with the same domains (preoperative PF to MHQ; ρ = 0.33; UE to BCTQ-Functional Status: ρ = 0.39; both p < 0.01).

CONCLUSIONS

The PROMIS UE and PI domains demonstrated slightly worse responsiveness than the MHQ and BCTQ domains that was nonetheless acceptable. The PROMIS PF domain was unresponsive. All three PROMIS domains correlated with the MHQ and BCTQ, but the PROMIS UE and PI domains had notably stronger correlations to the MHQ and BCTQ domains than the PF domain did. We feel that the PROMIS UE and PI can be used to evaluate the clinical outcomes of patients undergoing CTR, while also providing more robust insight into overall health status because they are general PROMs. However, we do not recommend the PROMIS PF for evaluating patients undergoing CTR.

LEVEL OF EVIDENCE

Level II, diagnostic study.

Do Patient Sociodemographic Factors Impact the PROMIS Scores Meeting the Patient-Acceptable Symptom State at the Initial Point of Care in Orthopaedic Foot and Ankle Patients?

BACKGROUND

Patient-reported outcome measures such as the Patient-Reported Outcomes Measurement Information System (PROMIS) allow surgeons to evaluate the most important outcomes to patients, including function, pain, and mental well-being. However, PROMIS does not provide surgeons with insight into whether patients are able to successfully cope with their level of physical and/or mental health limitations in day-to-day life; such understanding can be garnered using the Patient-acceptable Symptom State (PASS). It remains unclear whether or not the PASS status for a given patient and his or her health, as evaluated by PROMIS scores, differs based on sociodemographic factors; if it does, that could have important implications regarding interpretation of outcomes and fair delivery of care.

QUESTIONS/PURPOSES

In a tertiary-care foot and ankle practice, (1) Is the PASS associated with sociodemographic factors (age, gender, race, ethnicity, and income)? (2) Do PROMIS Physical Function (PF), Pain Interference (PI), and Depression scores differ based on income level? (3) Do PROMIS PF, PI, and Depression thresholds for the PASS differ based on income level?

METHODS

In this retrospective analysis of longitudinally obtained data, all patients with foot and ankle conditions who had new-patient visits (n = 2860) between February 2015 and December 2017 at a single tertiary academic medical center were asked to complete the PROMIS PF, PI, and Depression survey and answer the following single, validated, yes/no PASS question: "Taking into account all the activity you have during your daily life, your level of pain, and also your functional impairment, do you consider that the current state of your foot and ankle is satisfactory?" Of the 2860 new foot and ankle patient visits, 21 patient visits (0.4%) were removed initially because all four outcome measures were not completed. An additional 225 patient visits (8%) were removed because the patient chart did not contain enough information to accurately geocode them; 15 patients visits (0.5%) were removed because the census block group median income data were not available. Lastly, two patient visits (0.1%) were removed because they were duplicates. This left a total of 2597 of 2860 possible patients (91%) in our study sample who had completed all three PROMIS domains and answered the PASS question. Patient sociodemographic factors such as age, gender, race, and ethnicity were recorded. Using census block groups as part of a geocoding method, the income bracket for each patient was recorded. A chi-square analysis was used to determine whether sociodemographic factors were associated with different PASS rates, two-way ANOVA analyses with pairwise comparisons were used to determine if PROMIS scores differed by income bracket, and a receiver operating characteristic (ROC) curve analysis was performed to determine PASS thresholds for the PROMIS score by income bracket. The minimum clinically important difference (MCID) for PROMIS PF in the literature in foot and ankle patients ranges from about 7.9 to 13.2 using anchor-based approaches and 4.5 to 4.7 using the ½ SD, distribution-based method. The MCID for PROMIS PI in the literature in foot and ankle patients ranges from about 5.5 to 12.4 using anchor-based approaches and about 4.1 to 4.3 using the ½ SD, distribution-based method. Both were considered when evaluating our findings. Such MCID cutoffs for PROMIS Depression are not as well established in the foot and ankle literature. Significance was set a priori at p < 0.05.

RESULTS

The only sociodemographic factor associated with differences in the proportion of patients achieving PASS was age (15% [312 of 2036] of patients aged 18-64 years versus 11% [60 of 561] of patients aged ≥ 65 years; p = 0.006). PROMIS PF (45 ± 10 for the ≥ USD 100,000 bracket versus 40 ± 10 for the ≤ USD 24,999 bracket, mean difference 5 [95% CI 3 to 7]; p < 0.001), PI (57 ± 8 for ≥ USD 100,000 versus 63 ± 7 for ≤ USD 24,999, mean difference -6 [95% CI -7 to -4]; p < 0.001), and Depression (46 ± 8 for the ≥ USD 100,000 bracket versus 51 ± 11 for ≤ USD 24,999, mean difference -5 [95% CI -7 to -3]; p < 0.001) scores were better for patients in the highest income bracket compared with those in the lowest income bracket. For PROMIS PF, the difference falls within the score change range deemed clinically important when using a ½ SD, distribution-based approach but not when using an anchor-based approach; however, the score difference for PROMIS PI falls within the score change range deemed clinically important for both approaches. The PASS threshold of the PROMIS PF for the highest income bracket was near the mean for the US population (49), while the PASS threshold of the PROMIS PF for the lowest income bracket was more than one SD below the US population mean (39). Similarly, the PASS threshold of the PROMIS PI differed by 6 points when the lowest and highest income brackets were compared. PROMIS Depression was unable to discriminate the PASS.

CONCLUSIONS

Discussions about functional and pain goals may need to be a greater focus of clinic encounters in the elderly population to ensure that patients understand the risks and benefits of given treatment options at their advanced age. Further, when using PASS in clinical encounters to evaluate patient satisfaction and the ability to cope at different symptom and functionality levels, surgeons should consider income status and its relationship to PASS. This knowledge may help surgeons approach patients with a better idea of patient expectations and which level of symptoms and functionality is satisfactory; this information can assist in ensuring that each patient's health goal is included in shared decision-making discussions. A better understanding of why patients with different income levels are satisfied and able to cope at different symptom and functionality levels is warranted and may best be accomplished using an epidemiologic survey approach.

LEVEL OF EVIDENCE

Level III, diagnostic study.

Comparison of Generic, Musculoskeletal-Specific, and Foot and Ankle-Specific Outcome Measures Over Time in Tibial Plafond Fractures

BACKGROUND

This study performed a psychometric analysis assessing and comparing the responsiveness of the relevant components of a generic (Short Form-36 [SF36]), a musculoskeletal-specific (Short Musculoskeletal Functional Assessment [SMFA]), and a foot and ankle-specific (Foot and Ankle Outcome Score [FAOS]) outcome score when evaluating surgically treated tibial plafond fractures over time.

METHODS

Fifty-one patients were followed for 12 months after their tibial plafond fracture. Responsiveness, or the ability to detect clinical change in a disease, was evaluated through the standardized response mean (SRM), the proportion meeting a minimal clinically important difference (MCID), and floor and ceiling effects.

RESULTS

The SRM of the SF36-Physical Component Summary (PCS) was significantly greater than the SMFA-dysfunction index (DI) (P < .01) and FAOS-Activities of Daily Living (ADL) (P = .01) between baseline and 6 months, whereas the SRMs of only SF36-PCS and FAOS-ADL differed (P = .01) between 6 and 12 months. The proportion of patients achieving an MCID for SF36-PCS was higher than FAOS-ADL (P = .03) between baseline and 6 months and higher than SMFA-DI (P = .04) between 6 and 12 months. The FAOS-ADL showed substantial ceiling effects at baseline (88.2%) but much less at 6 months (5.9%) and 12 months (9.8%). Smaller ceiling effects were observed for the SMFA-DI (11.8%) at baseline, whereas none were observed for the SF36-PCS.

CONCLUSIONS

This study found that the SF36-PCS had greater responsiveness in assessing tibial plafond fractures compared to the SMFA-DI and FAOS-ADL, particularly in the first 6 months after surgery. In addition, limitations were revealed in the SMFA-DI and FAOS-ADL. This study illustrates the necessary diligence required for selection of outcome measures, as musculoskeletal and anatomy specific scores are not necessarily superior.

LEVEL OF EVIDENCE

Level II, prospective cohort study.

Minimal Clinically Important Differences for PROMIS Physical Function, Upper Extremity, and Pain Interference in Carpal Tunnel Release Using Region- and Condition-Specific PROM Tools

PURPOSE

Uncertainty exists about what change in Patient-Reported Outcomes Measurement Information System (PROMIS) scores represents a clinically relevant improvement (minimal clinically important difference [MCID]) in hand surgery care. Using a region-specific patient-reported outcome measure (PROM) (Michigan Hand Question [MHQ]) and a condition-specific PROM (Boston Carpal Tunnel Questionnaire [BCTQ]), MCID values were determined for PROMIS Physical Function (PF), Upper Extremity (UE), and Pain Interference (PI) computerized adaptive testing among patients undergoing carpal tunnel release (CTR).

METHODS

Patients undergoing CTR with a single surgeon from November 2014 to April 2017 were asked to complete the BCTQ, MHQ, and PROMIS PF, UE, and PI at each visit. Patients who had completed questionnaires both at a preoperative and either a 6-week or a 3-month postoperative visit were included. The PROMIS PF, UE, and PI MCID values were calculated using previously determined MCID estimates in the literature with both region- (ie, MHQ) and condition-specific (ie, BCTQ) PROM anchors. The PROMIS domain MCID estimates were also determined using the distribution-based method.

RESULTS

A total of 70 patients fit our inclusion criteria. Using MHQ Function and Pain, PROMIS UE, PF, and PI MCIDs were 6.3, 1.8, and -8.9, respectively. Using the average of the 2 BCTQ domains, PROMIS UE, PF, and PI MCIDs were 8.0, 2.8, and -9.7, respectively. Using the distribution-based method, PROMIS UE, PF, and PI MCIDs were 4.2, 2.7, and -4.1, respectively.

CONCLUSIONS

Using region- and condition-specific PROMs, we were able to provide MCID estimates of PROMIS UE, PF, and PI for patients undergoing CTR.

CLINICAL RELEVANCE

Estimating PROMIS UE, PF, and PI MCIDs in CTR using validated region- and condition-specific PROMs provides hand surgeons a way to evaluate CTR outcomes not previously described in the literature. Surgeons should understand that these values are only estimates and future work is needed to verify whether they reflect clinical improvement.

Determination of Minimum Clinically Important Difference (MCID) in Visual Analog Scale (VAS) Pain and Foot and Ankle Ability Measure (FAAM) Scores After Hallux Valgus Surgery

BACKGROUND

Minimum clinically important difference (MCID) defines a threshold when determining clinically significant treatment improvement. Visual analog scale (VAS) and Foot and Ankle Ability Measure activities of daily living (FAAM-ADL) are commonly used for measuring hallux valgus correction. This study aimed to determine MCID in VAS pain and FAAM-ADL scores for hallux valgus correction and additionally, to identify variables influencing achievement of the VAS pain MCID.

METHODS

Patients undergoing hallux valgus surgery were retrospectively included. VAS pain, FAAM-ADL, and pain satisfaction surveys were collected preoperatively and minimum 1-year postoperatively. Using a 6-point Likert-type pain satisfaction scale, patients reporting low postoperative satisfaction scores 1 through 3 were categorized as "dissatisfied," and high satisfaction scores 4 through six as "satisfied." One distribution-based method and 2 anchor-based methods were used to calculate MCID. Further, a logistic regression was calculated to determine if one group (defined by sex, pain satisfaction, preoperative VAS pain, concomitant lesser toe deformity correction, and specific hallux valgus correction procedure) had a greater likelihood of achieving the VAS pain MCID threshold. This study included 170 patients with postoperative follow-up averaging 23.6 months.

RESULTS

Calculated MCID scores ranged from 1.8 to 5.2 points for VAS pain and 11.1 to 22.7 points for FAAM-ADL. Moderate deformity correction with proximal first metatarsal osteotomy (Ludloff) (OR=2.236, P = .036) or severe deformity correction with first tarsometatarsal arthrodesis (Lapidus) (OR=3.145, P = .046); and higher preoperative pain scores (OR=1.045, P < .010) had significantly higher odds of meeting VAS pain MCID.

CONCLUSION

This study demonstrated MCID values that may indicate significant pain and function improvement after hallux valgus correction. Higher preoperative pain, and utilization of proximal metatarsal osteotomy or first tarsometatarsal arthrodesis for moderate or severe deformity correction resulted in significantly greater likelihood of reaching the VAS pain MCID than utilizing distal metatarsal and/or proximal phalanx osteotomy for mild deformity treatment.

LEVEL OF EVIDENCE

Level IV, validating outcome measures.