Evaluating the Effects of Five-star Composite Ratings of Nursing Homes

Public reporting of quality information through provider report cards is regarded as a compelling tool to improve health care quality by shifting consumers to providers of better quality, which in turn may motivate providers to compete on quality. As in other health care sectors, the empirical evidence on the effects of public reporting in the nursing home market has been mixed. The objective of this paper is to evaluate whether reporting simplified and composite ratings matter to consumers.

In 2008, Nursing Home Compare (NHC), a web-portal that publishes report cards for nursing homes, made significant changes to its reporting system. It went from publishing a series of clinical outcome measures, staffing measures, and deficiency citations to a composite rating in which nursing homes are assigned stars, from one star indicating quality "much below average" to five stars indicating "quality much above average." The new NHC website allows consumers to directly rank and compare nursing homes in a simple manner while still publishing all the indicators previously reported. The number of stars is determined by cut-offs from a continuous score largely based on deficiency inspections.

We use a regression discontinuity design to estimate changes in the number of admissions six months after the launch of the new system for nursing homes that are close to one of the four cut-off points. Data for this study come from three different sources. The first contains the unpublished underlying scores and thresholds that were used to determine the number of stars. The second source of data is the Online Survey Certification and Reporting (OSCAR), which contains information on baseline facility characteristics. Finally, the Minimum Data Set (MDS) contains detailed clinical assessment information collected at regular intervals for every resident in a certified nursing home.

Comparison of observed nursing home and patients baseline characteristics shows that nursing homes close to a cut-off point are comparable, even though there are substantial differences in observed characteristics by assigned number of stars. Furthermore, there is no evidence of manipulation of the assignment score. Our main results show that nursing homes that obtained an additional star gained more admissions after the changes were implemented, with effects depending on the star level. Admission gains are stronger for nursing homes that obtain either five versus four stars or two versus one star.

These results show that the way in which information is reported to consumers matters, and that simplified, composite ratings induces consumers to choose facilities of better reported quality.