Economic profiling of primary care physicians: consistency among risk-adjusted measures

Risk-adjustment methods for all-payer comparative performance reporting in Vermont

Background

As the emphasis in health reform shifts to value-based payments, especially through multi-payer initiatives supported by the U.S. Center for Medicare & Medicaid Innovation, and with the increasing availability of statewide all-payer claims databases, the need for an all-payer, “whole-population” approach to facilitate the reporting of utilization, cost, and quality measures has grown. However, given the disparities between the different populations served by Medicare, Medicaid, and commercial payers, risk-adjustment methods for addressing these differences in a single measure have been a challenge.

Methods

This study evaluated different levels of risk adjustment for primary care practice populations – from basic adjustments for age and gender to a more comprehensive “full model” risk-adjustment method that included additional demographic, payer, and health status factors. It applied risk adjustment to populations attributed to patient-centered medical homes (283,153 adult patients and 78,162 pediatric patients) in the state of Vermont that are part of the Blueprint for Health program. Risk-adjusted expenditure and utilization outcomes for calendar year 2014 were reported in 102 adult and 56 pediatric primary-care comparative practice profiles.

Results

Using total expenditures as the dependent variable for the adult population, the r² for the model adjusted for age and gender was 0.028. It increased to 0.265 with the additional adjustment for 3M Clinical Risk Groups and to 0.293 with the full model. For the adult population at the practice level, the no-adjustment model had the highest variation as measured by the coefficient of variation (18.5) compared to the age and gender model (14.8); the age, gender, and CRG model (13.0); and the full model (11.7). Similar results were found for the pediatric population practices.

Conclusions

Results indicate that more comprehensive risk-adjustment models are effective for comparing cost, utilization, and quality measures across multi-payer populations. Such evaluations will become more important for practices, many of which do not distinguish their patients by payer type, and for the implementation of incentive-based or alternative payment systems that depend on “whole-population” outcomes. In Vermont, providers, accountable care organizations, policymakers, and consumers have used Blueprint profiles to identify priorities and opportunities for improving care in their communities.

Understanding the Extent and Drivers of Interphysician Cost Variation for Spine Procedures

STUDY DESIGN

Retrospective analysis of billing data, medical records, and hospital cost data.

OBJECTIVE

To quantify intersurgeon variation for hospital costs of four spine procedures while adjusting for patient comorbidities and demographic factors.

SUMMARY OF BACKGROUND DATA

Spine care accounts for $90 billion in health care expenditures in the United States. Past findings demonstrate regional variation in surgery rates and high intersurgeon variation for anterior cervical discectomies/fusions. However, less has been done to examine intersurgeon variation in resource use across multiple procedures while adjusting for patient characteristics outside of a surgeon's control.

METHODS

We examined intersurgeon variation for 1241 elective spine procedures at one facility for 3 years. The procedures included 1 to 2 level cases of anterior cervical discectomies/fusions, posterior lumbar decompressions/fusions, posterior laminectomies, and lumbar discectomies. We isolated mean and median costs by surgeon and adjusted for patient demographics, comorbidities, and procedure types. Finally, we examined variation in subcategories such as instrumentation and inpatient stay costs to determine which contribute to total cost variation.

RESULTS

Unadjusted costs per surgeon varied by a factor of 1.32 to 1.81 between lowest and highest cost surgeon depending on procedure. After adjusting for patient features and procedure, variation was reduced to 1.31x. Of the seven surgeons who had sufficient patient volume, one was significantly less costly (-$1,462 per procedure) whereas three were significantly more costly than mean (+$685, +$839, +$702 per procedure). Intersurgeon differences in supply and operating room costs largely accounted for total variation, though actual drivers of variation were surgeon-specific.

CONCLUSION

Surgeons vary in average cost for spine procedures, though variation is more modest once adjusted for patient characteristics. Data on procedure-level variation should be discussed with individual surgeons to shift practice patterns. Finally, the comparison methodology can be applied to other procedures and specialties.

LEVEL OF EVIDENCE

4.

Performance profiling in primary care: does the choice of statistical model matter?

BACKGROUND

Profiling is increasingly being used to generate input for improvement efforts in health care. For these efforts to be successful, profiles must reflect true provider performance, requiring an appropriate statistical model. Sophisticated models are available to account for the specific features of performance data, but they may be difficult to use and explain to providers.

OBJECTIVE

To assess the influence of the statistical model on the performance profiles of primary care providers. Data Source. Administrative data (2006–2008) on 2.8 million members of a Dutch health insurer who were registered with 1 of 4396 general practitioners.

METHODS

Profiles are constructed for 6 quality measures and 5 resource use measures, controlling for differences in case mix. Models include ordinary least squares, generalized linear models, and multilevel models. Separately for each model, providers are ranked on z scores and classified as outlier if belonging to the 10% with the worst or best performance. The impact of the model is evaluated using the weighted kappa for rankings overall, percentage agreement on outlier designation, and changes in rankings over time.

RESULTS

Agreement among models was relatively high overall (kappa typically .0.85). Agreement on outlier designation was more variable and often below 80%, especially for high outliers. Rankings were more similar for processes than for outcomes and expenses. Agreement among annual rankings per model was low for all models.

CONCLUSIONS

Differences among models were relatively small, but the choice of statistical model did affect the rankings. In addition, most measures appear to be driven largely by chance, regardless of the model that is used. Profilers should pay careful attention to the choice of both the statistical model and the performance measures.

Structuring payment for medical homes

Despite widespread interest in the medical home model, there has been a lack of careful assessment of alternative methods to pay practices that serve as medical homes. This paper examines four specific payment approaches: enhanced fee-for-service payments for evaluation and management; additional codes for medical home activities within fee-for-service payments; per patient per month medical home payments to augment fee-for-service visit payments; and risk-adjusted, comprehensive per patient per month payments. Payment policies selected will affect both the adoption of the model and its longer-term evaluation. Evaluations of ongoing demonstrations should focus on payment design as well as on care--and cost.

Risk-adjusted payment and performance assessment for primary care

BACKGROUND

Many wish to change incentives for primary care practices through bundled population-based payments and substantial performance feedback and bonus payments. Recognizing patient differences in costs and outcomes is crucial, but customized risk adjustment for such purposes is underdeveloped.

RESEARCH DESIGN

Using MarketScan's claims-based data on 17.4 million commercially insured lives, we modeled bundled payment to support expected primary care activity levels (PCAL) and 9 patient outcomes for performance assessment. We evaluated models using 457,000 people assigned to 436 primary care physician panels, and among 13,000 people in a distinct multipayer medical home implementation with commercially insured, Medicare, and Medicaid patients.

METHODS

Each outcome is separately predicted from age, sex, and diagnoses. We define the PCAL outcome as a subset of all costs that proxies the bundled payment needed for comprehensive primary care. Other expected outcomes are used to establish targets against which actual performance can be fairly judged. We evaluate model performance using R(2)'s at patient and practice levels, and within policy-relevant subgroups.

RESULTS

The PCAL model explains 67% of variation in its outcome, performing well across diverse patient ages, payers, plan types, and provider specialties; it explains 72% of practice-level variation. In 9 performance measures, the outcome-specific models explain 17%-86% of variation at the practice level, often substantially outperforming a generic score like the one used for full capitation payments in Medicare: for example, with grouped R(2)'s of 47% versus 5% for predicting "prescriptions for antibiotics of concern."

CONCLUSIONS

Existing data can support the risk-adjusted bundled payment calculations and performance assessments needed to encourage desired transformations in primary care.

Computer-facilitated substance use screening and brief advice for teens in primary care: an international trial

OBJECTIVE

Primary care providers need effective strategies for substance use screening and brief counseling of adolescents. We examined the effects of a new computer-facilitated screening and provider brief advice (cSBA) system.

METHODS

We used a quasi-experimental, asynchronous study design in which each site served as its own control. From 2005 to 2008, 12- to 18-year-olds arriving for routine care at 9 medical offices in New England (n = 2096, 58% females) and 10 in Prague, Czech Republic (n = 589, 47% females) were recruited. Patients completed measurements only during the initial treatment-as-usual study phase. We then conducted 1-hour provider training, and initiated the cSBA phase. Before seeing the provider, all cSBA participants completed a computerized screen, and then viewed screening results, scientific information, and true-life stories illustrating substance use harms. Providers received screening results and "talking points" designed to prompt 2 to 3 minutes of brief advice. We examined alcohol and cannabis use, initiation, and cessation rates over the past 90 days at 3-month follow-up, and over the past 12 months at 12-month follow-up.

RESULTS

Compared with treatment as usual, cSBA patients reported less alcohol use at follow-up in New England (3-month rates 15.5% vs 22.9%, adjusted relative risk ratio [aRRR] = 0.54, 95% confidence interval 0.38-0.77; 12-month rates 29.3% vs 37.5%, aRRR = 0.73, 0.57-0.92), and less cannabis use in Prague (3-month rates 5.5% vs 9.8%, aRRR = 0.37, 0.17-0.77; 12-month rates 17.0% vs 28.7%, aRRR = 0.47, 0.32-0.71).

CONCLUSIONS

Computer-facilitated screening and provider brief advice appears promising for reducing substance use among adolescent primary care patients.

A nonparametric statistical method that improves physician cost of care analysis

OBJECTIVE

To develop a compositing method that demonstrates improved performance compared with commonly used tests for statistical analysis of physician cost of care data.

DATA SOURCE

Commercial preferred provider organization (PPO) claims data for internists from a large metropolitan area.

STUDY DESIGN

We created a nonparametric composite performance metric that maintains risk adjustment using the Wilcoxon rank-sum (WRS) test. We compared the resulting algorithm to the parametric observed-to-expected ratio, with and without a statistical test, for stability of physician cost ratings among different outlier trimming methods and across two partially overlapping time periods.

PRINCIPAL FINDINGS

The WRS algorithm showed significantly greater within-physician stability among several typical outlier trimming and capping methods. The algorithm also showed significantly greater within-physician stability when the same physicians were analyzed across time periods.

CONCLUSIONS

The nonparametric algorithm described is a more robust and more stable methodology for evaluating physician cost of care than commonly used observed-to-expected ratio techniques. Use of such an algorithm can improve physician cost assessment for important current applications such as public reporting, pay for performance, and tiered benefit design.

Who values information from a health plan Internet-based decision tool and why: a demographic and utilization analysis

OBJECTIVES

The aim of this study was to investigate factors associated with utilization of health plan Internet-based decision tools.

DATA SOURCES AND STUDY SETTING

Enrollment, claims, plan design, and web transaction data during 2008 provided by a national health insurer for 253,398 subscribers from 919 employers.

STUDY DESIGN

Multivariate models of the effects of demographic, health, employer, and plan benefit design characteristics on the use of the tool and its individual function categories.

DATA EXTRACTION METHODS

Subscribers, who were either an individual member or a family, were included if at least one family member had 12 months of coverage in 2008. Members older than 65 and those with multiple insurance carriers were excluded.

PRINCIPAL FINDINGS

Higher education, higher income, younger age, female gender, higher co-morbidity risk, prevalence of chronic conditions, Caucasian race, and English as the primary language were positively associated with using the tool. Plan benefit characteristics such as free preventive coverage, higher deductible, moderate coinsurance rate, family coverage, and enrollment in health savings accounts were also associated with higher likelihood of using the tool.

CONCLUSIONS

Insurers provide consumers information on cost efficiency, quality, and wellness through Internet-based decision tools, but more effort is needed to reach certain demographics.

Do differences in profiling criteria bias performance measurements? Economic profiling of medical clinics under the Korea National Health Insurance program: an observational study using claims data

Background

With a greater emphasis on cost containment in many health care systems, it has become common to evaluate each physician's relative resource use. This study explored the major factors that influence the economic performance rankings of medical clinics in the Korea National Health Insurance (NHI) program by assessing the consistency between cost-efficiency indices constructed using different profiling criteria.

Methods

Data on medical care benefit costs for outpatient care at medical clinics nationwide were collected from the NHI claims database. We calculated eight types of cost-efficiency index with different profiling criteria for each medical clinic and investigated the agreement between the decile rankings of each index pair using the weighted kappa statistic.

Results

The exclusion of pharmacy cost lowered agreement between rankings to the lowest level, and differences in case-mix classification also lowered agreement considerably.

Conclusions

A medical clinic may be identified as either cost-efficient or cost-inefficient, even when using the same index, depending on the profiling criteria applied. Whether a country has a single insurance or a multiple-insurer system, it is very important to have standardized profiling criteria for the consolidated management of health care costs.

Incorporating statistical uncertainty in the use of physician cost profiles

BACKGROUND

Physician cost profiles (also called efficiency or economic profiles) compare the costs of care provided by a physician to his or her peers. These profiles are increasingly being used as the basis for policy applications such as tiered physician networks. Tiers (low, average, high cost) are currently defined by health plans based on percentile cut-offs which do not account for statistical uncertainty. In this paper we compare the percentile cut-off method to another method, using statistical testing, for identifying high-cost or low-cost physicians.

METHODS

We created a claims dataset of 2004-2005 data from four Massachusetts health plans. We employed commercial software to create episodes of care and assigned responsibility for each episode to the physician with the highest proportion of professional costs. A physicians' cost profile was the ratio of the sum of observed costs divided by the sum of expected costs across all assigned episodes. We discuss a new method of measuring standard errors of physician cost profiles which can be used in statistical testing. We then assigned each physician to one of three cost categories (low, average, or high cost) using two methods, percentile cut-offs and a t-test (p-value < or = 0.05), and assessed the level of disagreement between the two methods.

RESULTS

Across the 8689 physicians in our sample, 29.5% of physicians were assigned a different cost category when comparing the percentile cut-off method and the t-test. This level of disagreement varied across specialties (17.4% gastroenterology to 45.8% vascular surgery).

CONCLUSIONS

Health plans and other payers should incorporate statistical uncertainty when they use physician cost-profiles to categorize physicians into low or high-cost tiers.

Development of a Medicaid behavioral health case-mix model

Many Medicaid programs have either fully or partially carved out mental health services. The evaluation of carved out plans requires a case-mix model that accounts for differing health status across Medicaid managed care plans. This article develops a diagnosis-based case-mix adjustment system specific to Medicaid behavioral health care. Several different model specifications are compared that use untransformed, square root transformed, and log-transformed expenditures.

A systematic review of health care efficiency measures

OBJECTIVE

To review and characterize existing health care efficiency measures in order to facilitate a common understanding about the adequacy of these methods.

DATA SOURCES

Review of the MedLine and EconLit databases for articles published from 1990 to 2008, as well as search of the "gray" literature for additional measures developed by private organizations.

STUDY DESIGN

We performed a systematic review for existing efficiency measures. We classified the efficiency measures by perspective, outputs, inputs, methods used, and reporting of scientific soundness.

PRINCIPAL FINDINGS

We identified 265 measures in the peer-reviewed literature and eight measures in the gray literature, with little overlap between the two sets of measures. Almost all of the measures did not explicitly consider the quality of care. Thus, if quality varies substantially across groups, which is likely in some cases, the measures reflect only the costs of care, not efficiency. Evidence on the measures' scientific soundness was mostly lacking: evidence on reliability or validity was reported for six measures (2.3 percent) and sensitivity analyses were reported for 67 measures (25.3 percent).

CONCLUSIONS

Efficiency measures have been subjected to few rigorous evaluations of reliability and validity, and methods of accounting for quality of care in efficiency measurement are not well developed at this time. Use of these measures without greater understanding of these issues is likely to engender resistance from providers and could lead to unintended consequences.

Regression tree boosting to adjust health care cost predictions for diagnostic mix

OBJECTIVE

To assess the ability of regression tree boosting to risk-adjust health care cost predictions, using diagnostic groups and demographic variables as inputs. Systems for risk-adjusting health care cost, described in the literature, have consistently employed deterministic models to account for interactions among diagnostic groups, simplifying their statistical representation, but sacrificing potentially useful information. An alternative is to use a statistical learning algorithm such as regression tree boosting that systematically searches the data for consequential interactions, which it automatically incorporates into a risk-adjustment model that is customized to the population under study.

DATA SOURCE

Administrative data for over 2 million enrollees in indemnity, preferred provider organization (PPO), and point-of-service (POS) plans from Thomson Medstat's Commercial Claims and Encounters database.

STUDY DESIGN

The Agency for Healthcare Research and Quality's Clinical Classification Software (CCS) was used to sort 2001 diagnoses into 260 diagnosis categories (DCs). For each plan type (indemnity, PPO, and POS), boosted regression trees and main effects linear models were fitted to predict concurrent (2001) and prospective (2002) total health care cost per patient, given DCs and demographic variables.

PRINCIPAL FINDINGS

Regression tree boosting explained 49.7-52.1 percent of concurrent cost variance and 15.2-17.7 percent of prospective cost variance in independent test samples. Corresponding results for main effects linear models were 42.5-47.6 percent and 14.2-16.6 percent.

CONCLUSIONS

The combination of regression tree boosting and a diagnostic grouping scheme, such as CCS, represents a competitive alternative to risk-adjustment systems that use complex deterministic models to account for interactions among diagnostic groups.

The chasm in neonatal outcomes in relation to time of birth in Lebanon

PURPOSE

The purpose of this study was to investigate the relationship between the time of birth and the mortality and morbidity of infants admitted to neonatal intensive care units.

DESIGN

This prospective, cohort study examined the records of women and infants admitted to the NICUs of four hospitals in Beirut, Lebanon, between July 1, 2002, and June 30, 2003. The hospitals selected were university affiliated and had a large number of deliveries (5,152 total for the year 2002-2003).

MAIN OUTCOME VARIABLES

Neonatal mortality and morbidity for infants admitted to the NICU were evaluated in relation to time of birth.

RESULTS

For the whole sample, mortality was higher for infants born during the night shift than for those born during the day shift. Mortality, morbidity, and brain asphyxia rates were also higher for infants born during the night shift and admitted to the NICU. Maternal risk factors and delivery complications were nor consistently higher on the night shift.

Assigning ambulatory patients and their physicians to hospitals: a method for obtaining population-based provider performance measurements

OBJECTIVE

To develop a method for assigning Medicare enrollees and the physicians who serve them to individual hospitals with adequate validity to allow population-based assessments of provider specific costs and quality of care.

DATA SOURCES/STUDY SETTING

The study populations consist of a 20 percent sample of Medicare fee-for-service enrollees and all physicians submitting claims for Medicare services from 1998 to 2000. Data were obtained from Medicare claims and enrollment files, Medicare's MPIER file, and from the American Hospital Association Annual Survey.

STUDY DESIGN

Cross-sectional analysis of the characteristics of hospitals, their extended medical staffs (EMSs) and the utilization patterns of their assigned Medicare enrollees.

DATA COLLECTION METHODS

Medicare enrollees were assigned to their predominant ambulatory physician and then to the hospital where that physician provided inpatient services or where a plurality of that physician's patient panel had medical admissions. Each beneficiary was linked to a physician and a hospital regardless of whether the patient was hospitalized creating Ambulatory Provider Specific Cohorts (APSCs).

PRINCIPAL FINDINGS

Ninety-six percent of eligible Medicare enrollees who had an index physician visit in 1998 were assigned to a specific provider. Two-thirds of the medical admissions during a 2-year period occurred at the assigned hospital and two-thirds of evaluation and management services were billed by the assigned hospital's EMS. The empirically derived EMS for hospitals had reasonable face and discriminant validity in terms of number and type of physicians practicing at different sized and type hospitals. Estimates of risk-adjusted costs across physician groups in year one are highly predictive of costs in a subsequent year (r=0.87, p<.0001 and weighted kappa=0.65, p<.0001).

CONCLUSIONS

Medicare claims data can be used to assign virtually all Medicare enrollees to empirically defined care systems comprised of hospitals and the physicians who practice at these hospitals. Studies of patterns of practice, costs and outcomes of care experienced by these APSCs could complement other methods of monitoring provider performance.

The effect of complementary and alternative medicine claims on risk adjustment

OBJECTIVE

We sought to assess how the inclusion of claims from complementary and alternative medicine (CAM) providers affects measures of morbidity burden and expectations of health care resource use for insured patients.

METHODS

Claims data from Washington State were used to create 2 versions of a case-mix index. One version included claims from all provider types; the second version omitted claims from CAM providers who are covered under commercial insurance. Expected resource use was also calculated. The distribution of expected and actual resource use was then compared for the 2 indices.

RESULTS

Inclusion of claims from CAM providers shifted 19,650 (32%) CAM users into higher morbidity categories. When morbidity categories were defined using claims from all providers, CAM users in the highest morbidity category had average (+/-SD) annual expenditures of $6661 (+/-$13,863). This was less than those in the highest morbidity category when CAM provider claims were not included in the index ($8562 +/- $16,354), and was also lower than the highest morbidity patients who did not use any CAM services ($8419 +/- $18,885).

CONCLUSIONS

Inclusion of services from CAM providers under third-party payment increases risk scores for their patients but expectations of costs for this group are lower than expected had costs been estimated based only on services from traditional providers. Risk adjustment indices may need recalibration when adding services from provider groups not included in the development of the index.

Profiling efficiency and quality of physician organizations in Medicare

This article presents a methodology for profiling the cost efficiency and quality of care of physician organizations (POs). The method is implemented for the Boston metropolitan area using 2002 Medicare claims. After adjustments for case mix and other factors, 4 of 30 organizations are identified with different than average efficiency Twenty-one of 30 organizations are identified with a different composite quality of care than average. Without changes in PO behavior, the gains from redirecting patients from lower to higher efficiency and quality providers are likely to be limited.

Should episode-based economic profiles be risk adjusted to account for differences in patients' health risks?

OBJECTIVE

To determine whether additional risk adjustment is necessary in economic profiling of physicians when claims data are already grouped into episodes of care, and to measure effects of risk adjustment on cost efficiency rankings of physicians.

DATA SOURCES

Four years of inpatient, outpatient, professional, and pharmacy claims data from a mixed model HMO.

STUDY DESIGN

Claims data were processed through Symmetry Health Data Systems' episode treatment group (ETG) grouper to define episodes of care and Symmetry's episode risk group (ERG) software to define measures of patients' health risk scores. For each episode type (ETG), ETG-mean expected costs were calculated as the mean costs of all episodes of that type, and risk-adjusted expected costs were calculated using three alternative risk model formulations.

DATA COLLECTION

Within specialties, physicians were ranked from most cost efficient to least cost efficient, based on standardized difference between actual and expected costs. ETG-mean based rankings were compared with risk-adjusted rankings. Analyses were performed for cardiologists, family practitioners, general surgeons, and neurologists.

PRINCIPAL FINDINGS

With all three risk models, risk scores were essentially unrelated to episode costs in approximately three-fourths of episode categories (ETGs). In a sample of ETGs for which risks-costs relationships appeared to exist, split sample validation showed the relationships to be unstable or spurious in all except one ETG. Within specialties, risk-adjusted cost efficiency rankings differ little from ETG-mean adjusted rankings.

CONCLUSIONS

Depending upon the purpose for which economic profiling is performed, additional risk adjustment, beyond that already provided by episode grouping, may be unnecessary. Additional research may be needed to identify and validate ETG-level relationships between patient risks and episode costs.