Generalization of the Difference-in-Differences Model and its Advantages for Applied Research

Introduction

Difference-in-differences (DD) models are common in health policy evaluation. They allow causal inference using observational data under the strong identifying assumption of parallel trends, i.e., treatment and control groups would have had parallel outcome trends in the absence of treatment.

We discuss one particular specification of triple difference (TD) models, which we call generalized difference-in-differences (GDD). GDD relaxes the parallel trends assumption by allowing the outcome trends of the treatment and control groups to differ up to a linear term.

Related models were introduced decades ago[1], inspired by Donald T. Campbell’s comparative interrupted time series (CITS) work.[2] Our conjecture is that these models were not widely applied by economists, despite their superiority to DD, due to the absence of an econometric specification that generalizes DD and a rigorous discussion of the required assumptions and formula to estimate the treatment effect. Very recently, Mora and Reggio[3] presented a fully flexible generalization of DD along with a discussion of the underlying assumptions and treatment effects. The aim of this paper is to present a specialization of their model that can be easily used by applied researchers to take full advantage of the relaxation of the parallel trends assumption. We compare the model to DD and explain its advantages.

Methodology

We build the GDD econometric specification (eq. 1) from the standard DD model by adding terms that control for differences in pre and post intervention trends for both the treatment and control groups.

Y=b₀+b₁*treat*post*time+b₂*treat*post+b₃*treat*time+b₄*post*time+b₅*treat+b₆*time+b₇*post+e (eq. 1)

where Y is the outcome of interest, treat is an indicator for treatment group, post is an indicator for post-treatment period, and time represents the time period. Terms in bold are additions to the standard DD model, which consists only of the terms including b₀, b₂, b₅, and b₇.

Then, we use simulations to show how point estimates from the GDD and DD models compare to each other, under different scenarios. We combine the simulations with theoretical insights to discuss the contribution of each added term in achieving a valid estimate. The point estimate for the intervention effect is calculated as

b₂+b₁*[(1+2+…+N)/N]

where N is the number of post-intervention periods.

Findings

GDD generalizes DD in the sense that its identifying assumption holds under less restrictive conditions. When the parallel trend assumption was satisfied, the GDD and DD models produced the same point estimates. When the trends of the treatment and comparison groups differed linearly at baseline, only GDD produced consistent estimates.

Implications

GDD relaxes the identifying assumption of DD at little cost. This is a strong reason for applied researchers to consider favoring GDD over DD for any application that satisfies the slightly longer pre-period requirement of the model.

References

[1] Simonton, D.K. (1977). Cross-sectional time-series experiments: some suggested statistical analyses. Psychological Bulletin 84(3), 489-502

[2] Campbell, DT and Stanley, JC. (1966). Experimental and quasi-experimental designs for research. Chicago: Rand McNally: 1966.

[3] Mora R. and Reggio I. (2017). Alternative diff-in-diffs estimators with several pretreatment periods. Econometric Reviews, 1-22