Sorting, Upcoding, and the Elusive Learning Curve: the Case of Coronary Artery Bypass Graft

This paper is motivated by two potential sources for incomplete risk adjustment when evaluating the quality of health care: (i) the endogenous sorting of patients based on unobservable characteristics, and (ii) the potential upcoding of observable patient characteristics used in risk adjustments. Both issues contaminate the estimation of the physician learning curve, which has important implications on the organization of health care delivery, payment, and performance evaluation.

In this paper, we gauge the impact of both sources of incomplete risk adjustment on the observed physician learning curve. We do so in the context of coronary artery bypass graft (CABG) surgery, which is particularly suitable for our purpose because of the procedure’s prevalence, importance, and surgical complexity. Our data covers the universe of inpatient discharge records from New York State, which has rich information at the case level and allows us to track individual physicians over time and across facilities.

We begin our empirical analysis by constructing patient risk scores in the same fashion as those used in the New York State Report Cards, and then proceed in three steps:

We first document the extent to which the prevailing risk adjustment method is confounded by patient sorting and risk factor upcoding. Using both case studies on some sample physicians and panel regression analysis, we find that a given physician’s case mix tends to have a lowerrisk score when the physician becomes more senior within a hospital, which is counter-intuitive and suggests the possibility of more upcoding of observed risk factors by less experienced physicians.

We then try to separately identify the causes of such patterns. We employ the following three sources of identification: that sorting based on unobservable patient characteristics is mitigated among emergency cases; that the change in a physician’s relative seniority within a hospital due to physician turnover is arguably exogenous; and that the incentive to upcode risk factors varies both across patients and over time. We find that the decline of risk scores with physician seniority is more prevalent among emergency cases, suggesting a strong possibility of upcoding. We also document cyclicality of risk scores within a calendar year: all else equal, risk scores are higher in July, August, September, and in December, when the incentive to upcode is likely stronger (to compensate for the lack of skill in a new cohort of residents that start in July, and for the possible under-staffing in the holiday season in December).

Finally, we compare the physician learning curves estimated using the current risk adjustment method with one that accounts for the above-noted issues. We use the physician’s in-hospital seniority as an instrument for unobserved patient sorting and potential risk factor upcoding. We find that the latter learning curve, estimated using the refined method, is on average 28% steeper than the former, suggesting room for improvement in the evaluation of physician performance and potentially the patient-physician matching mechanism prior to the delivery of health care.