Advances in quantitative methods for policy evaluation and comparative effectiveness research
The major methodological challenge in policy evaluations and comparative effectiveness research is unobserved confounding. Traditional approaches to this problem include difference-in-differences (DiD), and Instrumental Variable (IV) estimators. In this session, we consider advances in quantitative methods that make alternative assumptions about unobserved confounding. Ryan and Burgess consider settings where the expected outcomes for the treatment and control groups do not follow parallel trends. The authors find that propensity score matching prior to DiD estimation leads to less biased estimates than alternative approaches. O’Neill et al, consider the Generalised Synthetic Control (GSC) method. This approach allows unobserved confounders to have time-varying effects, by extending the synthetic control method with an interactive fixed effects (IFE) model. The paper examines the GSC approach in a re-analysis of Advancing Quality (AQ), an English hospital-based Pay for Performance scheme. Keele et al, consider near-far matching, which is a matched paired IV study design, which aims to strengthen the instrument, and balance characteristics between the comparison groups. The paper contrasts near-far matching with traditional IV methods in a case study evaluating the effect of reduced time to admission to intensive care on mortality. Throughout, the session will examine the implications of extending, and critically examining quantitative methods developed elsewhere, to needs of policy evaluation and comparative effectiveness research.