105
The Effect of Service Structure Improvement on Children's Mental Health: A Principal Stratification Approach
“Principal stratification” is recently developed in the statistical literature to infer causal relationships from observational data. The article illustrates how to apply principal stratification to health economics research. Specifically the general principal stratification framework is extended with a three-level mediator. An empirical illustration with data from the Fort Bragg Evaluation Project (FBEP) is provided to evaluate effect of improvement in the structure of service delivery beyond outpatient and inpatient on children’s mental health.
Background:
The Fort Bragg Evaluation Project (FBEP) was a quasi-experimental demonstration project designed to improve mental health outcomes by creating a new continuum of intermediate mental health services for children and adolescents referred for mental health treatment. The new intermediate services, including day treatment, in-home counseling, therapeutic foster homes, specialized group homes, and 24-hr crisis management services, were less restrictive than inpatient services but more intensive than outpatient. The intermediate services are available to the demonstration group but not the comparison group.
Method:
Principal stratification framework is extended with a mediator with the following three levels: inpatient, outpatient, and intermediate. Participants are classified into five latent groups according to service types they would use in demonstration or comparison groups. Four assumptions were implemented to reduce the unknown parameters and to identify the model. First, randomization assumption reflects the quasi-experimental design of the FEBP study. Children were treated as if they were randomly assigned. Secondly, it is assumed that moving to the demonstration from the comparison (or vice versa) would not move an individual between inpatient and outpatient services (skipping over intermediate services). Thirdly, the intervention had no effect other than by changing the level of services received. The last assumption is that there were no benefits of intermediate services for those who otherwise would have received outpatient services. Mental health outcome improvements were measured by changes of Child Behavior Checklist (CBCL) scores. Bootstrapping method is used to calculate all confidence intervals. Two sensitivity analyses were performed when two alternative assumptions were set as the last assumption: scenario (2) assumed Intermediate services are harmful for youth who would have been treated in outpatient settings at the comparison, and scenario (3) assumed the percentage of individuals who received intermediate services would have received inpatient services as 100%.
Results/Discussion:
The result shows that most of the patients who moved into intermediate moved down from inpatient services. Although this group did show a significant improvement in their mental health outcome, the 95% confidence interval for the effect includes zero in except in scenario (3.) Overall this article presents a new analysis of the children’s mental health service with principal stratification framework and demonstrated that such analyses might be sensitive to assumptions. The extensions of principal stratification method described in this article also provide useful areas of future research.