Posterior Predictive Outlier Detection Using Sample Reweighting

Bayesian hierarchical modeling of substate area estimates from the Medicare CAHPS survey

Each year, surveys are conducted to assess the quality of care for Medicare beneficiaries, using instruments from the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) program. Currently, survey measures presented for Fee-for-Service beneficiaries are either pooled at the state level or unpooled for smaller substate areas nested within the state; the choice in each state is based on statistical tests of measure heterogeneity across areas within state. We fit spatial-temporal Bayesian random-effects models using a flexible parameterization to estimate mean scores for each of the domains formed by 94 areas in 32 states measured over 5 years. A Bayesian hat matrix provides a heuristic interpretation of the way the model combines information for estimates in these domains. The model can be used to choose between reporting of state- or substate-level direct estimates in each state, or as a source of alternative small-area estimates superior to either direct estimate. We compare several candidate models using log pseudomarginal likelihood and posterior predictive checks. Results from the best-performing model for 8 measures surveyed from 2012 to 2016 show substantial reductions in mean squared error (MSE) over direct estimates.