Benchmarking Patient Safety and Quality in U.S. Hospitals: The Stochastic Frontier Approach

Research Objective: Paying for quality, not paying for serious adverse events, and public reporting of quality are used by public and private payers to incentivize hospitals to improve performance. Quality benchmarks of the Centers for Medicaid and Medicare Services’ (CMS) are national average scores of quality and safety indicators. In the on-line “Hospital Compare,” CMS ranks providers according to whether they are above, at, or below the national average on a number of indicators. However, there is no evidence that an average is a good indicator by which to measure the quality of hospital care or to compare care across hospitals. We use stochastic frontier (SF) analysis to: 1) develop scientifically-derived “best practice” outcomes; 2) calculate gaps between them and observed outcomes for several of the CMS Hospital Quality Initiatives outcomes measures; 3) ascertain the distributions of these gaps in a large national sample of U.S. hospitals; and 4) determine the factors that could be changed in order to reduce those gaps.

Methods: The sample was all hospitals that matched on analysis variables from the following data sources: American Hospital Association Annual Survey, CMS cost reports, Area Resource File, and the CMS RHQDAPU file (1,823 – 2,784). Truncated normal SF models with bootstrapping for heteroskedasticity were estimated. SF model outputs were the best possible outcomes for each hospital and the gaps between those outcomes and the hospital’s observed outcomes. Gap distributions were calculated.

Results: The results varied across the several CMS outcome measures. For instance, the top 5% of hospitals had gaps of 0-9% (depending on which outcome is examined.) The top 10% of hospitals had gaps of 0-15%. Analysis of the distribution of gaps shows that 52 -94% of hospitals had > 10% quality and safety gaps while 0-15% of hospitals had > 50% quality and safety gaps. Profit margin, ownership and percent of for-profit hospitals in the region were not strong predictors of gaps. Payer mix, RN staffing, size, case mix index, accreditation, being a teaching hospital, market share, urban location, and region were strong predictors of gaps.