Evaluations of early childhood programmes have played an important role in shaping policy debates on early education. For instance, the Head Start Impact Study, a recent randomised control trial of Head Start, reported small effect sizes that fade considerably over a few years (Puma et al. 2010, 2012). These findings have heightened debate among academics over the cost effectiveness of the Head Start programme (e.g. Barnett 2011, Gibbs et al. 2013, Kline and Walters,2016) and have frequently been cited by critics who argue that Head Start is ineffective in achieving its mission and should be abandoned or seriously reformed. Given the policy impact of the findings from early education interventions, and more broadly any social intervention, accurate evaluation of the total effect of these programmes is crucial.
This is what we do in a recent paper (List et al. 2019) by providing the first empirical evidence on spillover effects from a large-scale early education intervention by causally estimating neighbour effects. We find that ignoring these effects results in severe underestimation of the total impact.
Between 2010 and 2014, a series of early childhood interventions were delivered to low-income families with young children in Chicago Heights Early Childhood Center (CHECC). The centre was located in Chicago Heights, IL, which is a neighbourhood in the South Side area of Chicago with characteristics similar to many other low-performing urban school districts. According to the 2010 Census, black and Hispanic minorities constituted about 80% of the population of Chicago Heights, its per-capita income was $17,546 per year, and 90% of students attending the Chicago Heights School District were receiving free or reduced-price lunches.
The goals of the intervention were to examine how investing in cognitive and non-cognitive skills of low-income children aged three to four affects their short- and long-term outcomes, and to evaluate the effectiveness of investing directly in the child's education versus indirectly through the parents. To that end, families of over 2,000 disadvantaged children were randomised into either (i) an incentivized parent-education programme (Parent Academy), (ii) a high-quality preschool program (Pre-K), or (iii) a control group. The children's cognitive and non-cognitive skills were assessed on a regular basis, starting before the randomisation and continuing into the middle and end of the programmes. Follow-up assessments were also conducted on a yearly basis.
To estimate the spatial spillover effects from the intervention, we calculate the number of treated neighbours of a child at a given time and use it as a measure of spatial exposure to treatments. To do so, we start by calculating commuting distances between the home locations of all pairs of children who were randomised during the intervention. We define a pair as neighbours if the commuting distance between the two is less than r kilometres, and we call r the neighbourhood radius. Figure 1 presents a histogram of travel distances between home locations of all children who were randomized during the intervention
We then estimate the impact of the number of i’s neighbours residing within r kilometres who are randomised into treatments – i.e. the spatial exposure rate of individual i who is not treated – on the standardized cognitive or non-cognitive test score of a child i on test t. We control for the total number of i’s CHECC neighbours residing within r kilometres at a given point in time. We also include individual and test (time) fixed effects. We estimate spillover effects for the neighbourhood radii of 3, 5, and 7 kilometres.
We find significant positive spillover effects on both cognitive and non-cognitive test scores. The effects on non-cognitive scores are more than double the effects on cognitive scores: an additional treated neighbour within 3 kilometres of a child's home increases her cognitive score by 0.0033σ, whereas it increases her non-cognitive score by 0.0069σ. We also show that the spillover effects from an additional treated neighbour becomes smaller as we broaden the neighbourhood radius from 3 to 7 kilometres (see Figure 2).
Since African Americans and Hispanics make up over 90% of our sample, our analysis on heterogeneity along race focuses on these two groups. We find that, on average, an additional treated neighbour increases the non-cognitive scores of an African American child by about two to three times as much as a Hispanic child. We also look at gender effects and show that, on average, boys benefit more than girls from both cognitive and non-cognitive spillovers.
Our evidence indicates that non-cognitive spillovers are more likely to operate through children’s rather than parents’ social networks. We also find suggestive evidence that cognitive spillover effects are generated through influencing the parents' decision to enrol their child in an alternative form of treatment in an outside preschool programme.
More generally, our results indicate that interventions that promote social interactions both within participants and between participants and non-participants are likely to generate larger positive externalities on non-cognitive skills. Our findings also suggest that traditional measures of early education impacts, which ignore externalities, are likely to be too pessimistic when such programs are taken to scale. Thus, ignoring the spillover effects can result in a severe underestimation of the programme impact, leading to fewer programmes being taken to scale than is optimal.
Barnett, W S (2011), “Effectiveness of early educational intervention,” Science 333: 975-978.
Gibbs, C, J Ludwig and D Miller (2013), “Does Head Start do any lasting good?”, in M J Bailey and S Danziger (eds), Legacies of the War on Poverty, Russell Sage Foundation Press, pp. 39-65.
Kline, P and C R Walters (2016), “Evaluating public programs with close substitutes: The case of Head Start,” Quarterly Journal of Economics 131(4): 1795-1848.
List, J, F Momeni and Y Zenou (2019) “Are Estimates of Early Education Programs Too Pessimistic? Evidence from a Large-Scale Field Experiment that Causally Measures Neighbor Effects”, CEPR Discussion Paper 13725.
Puma, M, S Bell, R Cook and C. Heid (2010), Head Start Impact Study: Final Report, U.S. Department of Health and Services, Administration for Children and Families.
Puma, M, S Bell, R Cook, C Heid, P Broene, F Jenkins, A Mashburn and J. Downer (2012), Third Grade Follow-up to the Head Start Impact Study Final Report, OPRE Report No. 2012-45, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.