Evaluation of Reliability and Correlations of Quality Measures in Cancer Care

The reliability and validity of lung cancer and melanoma clinical quality survival measures

OBJECTIVE

To develop a risk adjustment approach and test reliability and validity for oncology survival measures.

DATA SOURCES AND STUDY SETTING

We used the National Cancer Institute's Surveillance, Epidemiology, and End Results (SEER)-Medicare data from 2010 to 2013, with mortality data through 2015.

STUDY DESIGN

We developed 2-year risk-standardized survival rates (RSSR) for melanoma, non-small cell lung cancer (NSCLC), and small cell lung cancer (SCLC). Patients were attributed to group practices based on the plurality of visits. We identified the risk-adjustment variables via bootstrap and calculated the RSSRs. Reliability was tested via three approaches: (1) signal-to-noise ratio (SNR) reliability, (2) split-half, and (3) test-retest using bootstrap. We tested known group validity by stage at diagnosis using Cohen's d.

DATA COLLECTION/EXTRACTION METHODS

We selected all patients enrolled in Medicare and linked to SEER during the measurement period with an incident first primary diagnosis of stage I-IV melanoma, NSCLC, or SCLC. We excluded patients with missing data on month and/or stage of diagnosis.

PRINCIPAL FINDINGS

Results are based on patients with melanoma (n = 4344); NSCLC (n = 16,080); and SCLC (n = 2807) diagnosed between 2012 and 2013. The median (interquartile range) for the RSSRs at the group practice-level were 0.89 (0.83-0.87) for melanoma, 0.37 (0.30-0.43) for NSCLC, and 0.19 (0.11-0.25) for SCLC. C-statistics for the models ranged from 0.725 to 0.825. The reliability varied by approach with median SNR 0.20, 0.25, and 0.13; median test-retest 0.59, 0.57, and 0.56; median split-half reliability 0.21, 0.29, and 0.29 for melanoma, NSCLC, and SCLC, respectively. Cohen's d for stage I-IIIa and IIIb+ was 1.27, 0.86, 0.60 for melanoma, NSCLC, and SCLC, respectively.

CONCLUSIONS

Our results suggest that these cancer survival measures demonstrated adequate test-retest reliability and expected findings for the known-group validity analysis. If data limitations and feasibility challenges can be addressed, implementation of these quality measures may provide a survival metric used for oncology quality improvement efforts.

Quality of Care for Prostate Cancer in Kashmir, India: A Real-World Study

Purpose Despite the importance of quality care for patients with prostate cancer, significant gaps exist in healthcare delivery, including diagnosis and treatment. Our objective was to assess the quality of care (QoC) using retrospective data from prostate care patients in our center. Methods We performed a retrospective study of prostate cancer patients registered at a dedicated cancer care center in the Kashmir region from 2012 to 2020. A set of 15 quality indicators representing crucial facets of diagnosis, pathology, and treatment was identified from a comprehensive list developed and validated by other researchers. Results The final analysis of all indicators was conducted on 46 patients with a median age of 70 years (52-92 years). In the majority of patients, the diagnosis (89.1%) was made through a prostatic biopsy, while only five patients were diagnosed solely based on the prostate-specific antigen. Transrectal ultrasound (TRUS) or transurethral resection (TURP)-guided biopsy was documented in 84.8% of patients, with Gleason grading documented in 87.5% of patients. However, the number of positive cores was mentioned for only 25.7% of patients. Radical prostatectomy was the primary treatment for most patients with localized prostate cancer (58.3%). The majority of patients with metastatic prostate cancer were treated with orchidectomy (55%), owing to easy access and the lower cost of surgical castration. Conclusion The study demonstrated a lack of compliance with many QoC indicators at the diagnostic and therapeutic levels. However, large-scale, population-based studies are needed to establish the compliance of prostate cancer QoC in Kashmir. The quality indicator assessment can guide the necessary actions required to improve QoC for prostate cancer patients.

Measuring variation in the quality of systemic anti-cancer therapy delivery across hospitals: A national population-based evaluation

AIM

To date, there has been little systematic assessment of the quality of care associated with systemic anti-cancer therapy (SACT) delivery across national healthcare systems. We evaluated hospital-level toxicity rates during SACT treatment as a means of identifying variation in care quality.

METHODS

All colorectal cancer (CRC) patients receiving SACT within 106 English National Health Service (NHS) hospitals between 2016 and 2019 were included. Severe acute toxicity rates were derived from hospital administrative data using a validated coding framework. Variation in hospital-level toxicity rates was assessed separately in the adjuvant and metastatic settings. Toxicity rates were adjusted for age, sex, comorbidity, performance status, tumour site, and TNM staging.

RESULTS

Eight thousand one hundred and seventy three patients received SACT in the adjuvant setting, and 7,683 patients in the metastatic setting. Adjusted severe acute toxicity rates varied between hospitals from 11% to 49% for the adjuvant cohort, and from 25% to 67% for the metastatic cohort. Compared to the national mean toxicity rate in the adjuvant cohort, six hospitals were more than two standard deviations (2SD) above, and four hospitals were more than 2SD below. In the metastatic cohort, six hospitals were more than 2SD above, and seven hospitals were more than 2SD below the national mean toxicity rate. Overall, 12 hospitals (12%) had toxicity rates more than 2SD above the national mean, and 11 (10%) had rates more than 2SD below.

CONCLUSION

There is substantial variation in hospital-level severe acute toxicity rates in both the adjuvant and metastatic settings, despite risk-adjustment. Ongoing reporting of this performance indicator can be used to focus further investigation of toxicity rates and stimulate quality improvement initiatives to improve care.

Practice Billing for Medicare Advance Care Planning Across the USA

BACKGROUND

Medicare introduced billing codes in 2016 to encourage clinicians to engage in advance care planning (ACP) and promote goal-concordantend-of-life care, but uptake has been modest. While prior research examined individual-level factors in ACP billing, organization-level factors associated with physician practices billing for ACP remain unknown.

OBJECTIVE

Examine the role of practices in ACP billing.

DESIGN

Retrospective cohort study analyzing 2016-2018 national Medicare data.

PARTICIPANTS

A total of 53,926 practices with at least 10 attributed Medicare beneficiaries.

MAIN MEASURES

Outcomes were practice-level ACP billing (any use by the practice) and ACP use rate by practice-attributed beneficiaries. Practice characteristics were number of beneficiaries attributed to the practice; percentage of beneficiaries by race, Medicare-Medicaid dual enrollment, sex, and age; practice size; and specialty mix.

KEY RESULTS

Fifteen percent of practices billed for ACP. In adjusted models, we found higher odds of ACP billing and higher ACP use rates among practices with more primary care physicians (billing AOR: 10.01, 95%CI: 8.81-11.38 for practices with 75-100% (vs 0) primary care physicians), and those serving more Medicare beneficiaries (billing AOR: 4.55, 95%CI 4.08-5.08 for practices with highest (vs lowest) quintile of beneficiaries), and larger shares of female beneficiaries (billing AOR: 3.06, 95% CI 2.01-4.67 for 75-100% (vs <25%) female ).

CONCLUSIONS

Several years after Medicare introduced ACP reimbursements for physicians, relatively few practices bill for ACP. ACP billing was more likely in large practices with a greater percentage of primary care physicians. To increase ACP billing uptake, policymakers and health system leaders might target interventions to larger practices where a small number of physicians already bill for ACP and to specialty practices that serve as the primary source of care for seriously ill patients.

Baseline cardiac function checkup in patients with gastric or breast cancer receiving trastuzumab or anthracyclines

BACKGROUND

Although trastuzumab and anthracyclines are frequently used to treat breast cancer (BC) and gastric cancer (GC), cardiotoxicity is a serious concern. The cardiac function assessment is recommended at baseline before initiating treatment. However, the prevalence rates of baseline cardiac checkups are unknown.

METHODS

The national database of hospital-based cancer registries linked to the health services-utilization data was used to study patients with newly diagnosed stage IV BC and GC (n = 6271) who received trastuzumab (n = 4324, 69.0%) or anthracyclines between January 2012 and December 2015. The baseline ultrasound echocardiogram (UCG) performance rate and factors related to adequate UCG performance for all patients and those receiving trastuzumab were analyzed.

RESULTS

The adequate baseline UCG checkup rate was higher in patients treated with trastuzumab than in those treated with anthracyclines (71.8% vs 44.1%, respectively). Additionally, patients with GC were less likely to receive an adequate baseline UCG performance than those with BC (70.4% vs 75.0%, respectively). After adjusting for potential confounders, patients with anthracycline-treated BC and GC were less likely to receive adequate baseline UCG performance than those with trastuzumab-treated BC (odds ratio [OR]: 0.24, 95% confidence interval [CI]: 0.20-0.28, and OR: 0.07, 95% CI: 0.03-0.16, respectively). Furthermore, patients with trastuzumab-treated GC were less likely to receive adequate baseline UCG performance than those with BC (OR: 0.65, 95% CI: 0.50-0.84).

CONCLUSIONS

The baseline UCG was less likely to be performed in patients receiving anthracyclines than in those receiving trastuzumab, as well as in patients with GC than in those with BC.

Hematology/Oncology Fellowship Programs' Participation in the Quality Oncology Practice Initiative

PURPOSE

In the first decade of this millennium, ASCO pioneered a quality measurement tool, the Quality Oncology Practice Initiative (QOPI). Despite an Accreditation Council for Graduate Medical Education (ACGME) requirement since 2012 for oncology fellows to participate in quality improvement (QI) projects, the uptake of QOPI remains modest.

METHODS

This study examined reasons for low QOPI participation by surveying participating and nonparticipating HemOnc Fellowship Programs. The survey elicited views toward QI and QOPI as well as ideas about making the program more helpful.

RESULTS

Among 69 fellowship programs, only 39% (n = 27) participated in QOPI. Other findings were that (1) the majority of programs considered their fellows' QI projects beneficial but were not fulfilling the ACGME standard for all fellows' QI participation; (2) nonparticipating programs were unfamiliar with but interested in QOPI; (3) participating programs tended to view QI as easier to conduct and more beneficial than nonparticipating programs; and (4) programs that withdrew from QOPI and participating programs alike were dissatisfied with the educational benefit and data abstraction burden for fellows.

CONCLUSION

Academic oncology programs generally valued QI but many have not fully engaged in it. Fellows in programs participating in QOPI may have had less difficulty conducting QI and their projects may have been more beneficial than that of nonparticipating programs. However, perceived lack of educational benefits for fellows and the burden of manual data abstraction from the electronic medical record are impediments to satisfaction with the program. Higher faculty involvement and longitudinal reports for each fellow may significantly increase participation.

Measuring Racial Inequities In The Quality Of Care Across Oncology Practices In The US

Racial inequities in clinical performance diminish overall health care system performance; however, quality assessments have rarely incorporated reliable measures of racial inequities. We studied care for more than one million Medicare fee-for-service beneficiaries with cancer to assess the feasibility of calculating reliable practice-level measures of racial inequities in chemotherapy-associated emergency department (ED) visits and hospitalizations. Specifically, we used hierarchical models to estimate adjusted practice-level Black-White differences in these events and described differences across practices. We calculated reliable inequity measures for 426 and 322 practices, depending on the measure. These practices reflected fewer than 10 percent of practices treating Medicare beneficiaries with chemotherapy, but they treated approximately half of all White and Black Medicare beneficiaries receiving chemotherapy and two-thirds of Black Medicare beneficiaries receiving chemotherapy. Black patients experienced chemotherapy-associated ED visits and hospitalizations at higher rates (54.2 percent and 35.8 percent, respectively) than White patients (45.7 percent and 31.9 percent, respectively). The median within-practice Black-White difference was 8.1 percentage points for chemotherapy-associated ED visits and 2.7 percentage points for chemotherapy-associated hospitalizations. Additional research is needed to identify other reliable measures of racial inequities in health care quality, measure care inequities in smaller practices, and assess whether providing practice-level feedback could improve equity.

Evaluating Quality Indicators of Glioblastoma Care: Audit Results From an Indian Tertiary Care Cancer Center

PURPOSE

There are limited reports of quality metrics in glioblastoma. We audited our adherence to quality indicators as proposed in the PRIME Quality Improvement study.

METHODS

This is a retrospective audit of patients treated between 2017 and 2020. After postsurgical integrated diagnosis, patients received radiotherapy (RT) with concurrent and adjuvant temozolomide (TMZ). Multiparametric magnetic resonance imaging at predefined times guided management. Numbers with proportions for indices were calculated. Survival was estimated using the Kaplan-Meier method.

RESULTS

One hundred six patients were consecutively treated. The median age was 55 years (interquartile range of 47-61 years) with a male preponderance (68%). Ninety-six (90.6%) patients underwent subtotal resection, and 10 (9.4%) biopsy alone. Isocitrate dehydrogenase was wild-type in 96 (91%), and O⁶-methylguanine-DNA methyltransferase was unmethylated in 70 (66.0%) patients. Telomerase reverse transcriptase promoter was mutated in 64 (60.4%), and TP53 was mutated in 22 (20.8%). Concurrent radiation and TMZ were planned for 104 (98.1%), and radiation alone for 2 (1.9%). The median time to concurrent RT-TMZ was 36 days (interquartile range 30-44 days). All patients planned for RT-TMZ completed treatment, but only 81 (76%) completed adjuvant TMZ. Sixty-three (59%) completed six cycles, 18 (17%) received less than six cycles, and 25 (24%) did not receive adjuvant TMZ. At a median follow-up of 24 months (range 21-31 months), the median (95% CI) progression-free survival and overall survival were 11 (95% CI, 9.4 to 13.0) and 20.0 (95% CI, 15 to 26) months, respectively.

CONCLUSION

Our patients met quality indices in most domains; outcomes are comparable with global results. Metrics will be periodically evaluated to include new standards and assess continuous service appropriateness.

Low-Value Care at the Actionable Level of Individual Health Systems

IMPORTANCE

Low-value health care remains prevalent in the US despite decades of work to measure and reduce such care. Efforts have been only modestly effective in part because the measurement of low-value care has largely been restricted to the national or regional level, limiting actionability.

OBJECTIVES

To measure and report low-value care use across and within individual health systems and identify system characteristics associated with higher use using Medicare administrative data.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study of health system-attributed Medicare beneficiaries was conducted among 556 health systems in the Agency for Healthcare Research and Quality Compendium of US Health Systems and included system-attributed beneficiaries who were older than 65 years, continuously enrolled in Medicare Parts A and B for at least 12 months in 2016 or 2017, and eligible for specific low-value services. Statistical analysis was conducted from January 26 to July 15, 2021.

MAIN OUTCOMES AND MEASURES

Use of 41 individual low-value services and a composite measure of the 28 most common services among system-attributed beneficiaries, standardized to distance from the mean value. Measures were based on the Milliman MedInsight Health Waste Calculator and published claims-based definitions.

RESULTS

Across 556 health systems serving a total of 11 637 763 beneficiaries, the mean (SD) use of each of the 41 low-value services ranged from 0% (0.01%) to 28% (4%) of eligible beneficiaries. The most common low-value services were preoperative laboratory testing (mean [SD] rate, 28% [4%] of eligible beneficiaries), prostate-specific antigen testing in men older than 70 years (mean [SD] rate, 27% [8%]), and use of antipsychotic medications in patients with dementia (mean [SD] rate, 24% [8%]). In multivariable analysis, the health system characteristics associated with higher use of low-value care were smaller proportion of primary care physicians (adjusted composite score, 0.15 [95% CI, 0.04-0.26] for systems with less than the median percentage of primary care physicians vs -0.16 [95% CI, -0.27 to -0.05] for those with more than the median percentage of primary care physicians; P < .001), no major teaching hospital (adjusted composite, 0.10 [95% CI, -0.01 to 0.20] without a teaching hospital vs -0.18 [95% CI, -0.34 to -0.02] with a teaching hospital; P = .01), larger proportion of non-White patients (adjusted composite, 0.15 [95% CI, -0.02 to 0.32] for systems with >20% of non-White beneficiaries vs -0.06 [95% CI, -0.16 to 0.03] for systems with ≤20% of non-White beneficiaries; P = .04), headquartered in the South or West (adjusted composite, 0.28 [95% CI, 0.14-0.43] for the South and 0.22 [95% CI, 0.02-0.42] for the West compared with -0.09 [95% CI, -0.26 to 0.08] for the Northeast and -0.44 [95% CI, -0.60 to -0.28] for the Midwest; P < .001), and serving areas with more health care spending (adjusted composite, 0.23 [95% CI, 0.11-0.35] for areas above the median level of spending vs -0.24 [95% CI, -0.36 to -0.12] for areas below the median level of spending; P < .001).

CONCLUSIONS AND RELEVANCE

The findings of this large cohort study suggest that system-level measurement and reporting of specific low-value services is feasible, enables cross-system comparisons, and reveals a broad range of low-value care use.