A Unified Approach to Implementation and Evaluation of CMS's Innovative Shared Savings Initiatives

Section 3021 of the Affordable Care Act established the Center for Medicare and Medicaid Innovation (CMMI) within the Centers for Medicare and Medicaid Services (CMS). CMMI tests innovative payment and service delivery models intended to reduce expenditures while improving the quality of care provided to beneficiaries. The Secretary of HHS is authorized to scale up successful models.

CMMI’s major models enable participating providers to earn additional payments if they produce reductions in Medicare expenditures while achieving minimum quality standards. These models fall into two broad categories, shared savings models (SSM) and bundled payments models, both of which aim to align participating providers’ incentives. Several of these models have been and will be tested in the coming years. These include, Pioneer ACO, MSSP, the CEC Model, BPCI, and the Oncology Care Model. The testing involves two stages: the implementation and the evaluation of each innovative model.

This two-staged testing process can lead to inconsistencies such as CMS paying shared savings incentives during the implementation stage and the evaluation showing no intervention effect for the same participants. This study develops a unified approach to implementation and evaluation that eliminates these inconsistencies, reduces testing costs, and accelerates the scale up.

SSM implementation involves three steps. First, a baseline outcome, typically per beneficiary per year (PBPY) cost of care is created for each participant using pre-model data. Second, the baseline PBPY cost is risk adjusted and trended forward to produce a PBPY benchmark expenditure. Third, the performance year (PY) PBPY is compared to the benchmark. If PY expenditures are lower, the ACO produced savings that are split between the payer and the ACO (shared savings). Otherwise, the ACO produced no savings and may be liable to pay CMS a portion of the difference between PY expenditures and benchmark in the form of shared losses.

Although the evaluation approach varies, it generally is a difference-in-differences regression-based analysis. In this approach, the pre-post change in PBPY cost of the ACOs is compared against the pre-post change in PBPY cost of a comparison group. If ACOs reduced costs more than the comparison group, the model is considered successful in reducing expenditures.

We present the general equation used to implement these innovative models. We also present the general difference-in-differences econometric equation used to evaluate the effect of these models. We compare the two equations and discuss differences and similarities. Although our discussion applies both to SSM and bundled payments, we illustrate with the case of SSM.

The implementation equation we develop can be specialized to accommodate different variations of SSM. We show that the implementation formula is conceptually similar to a simple difference-in-differences arithmetic analysis. We propose using regression based difference-in-difference analysis to implement and evaluate the models in a one-stage unified approach. The unified approach to implementation and testing reduces costs, eliminates inconsistencies, and accelerates the scaling up of successful models.