[Warning: this post has a high nerd-factor, read at your own risk]
The recent paper by Angrist et al. „The Elite Illusion: Achievement Effects at Boston and New York Exam Schools“ has received some attention recently, among others by Felix Salmon and Jesse Anttila-Hughes. It shows that attending an elite school does not increase your achievements.
I want to briefly discuss one interesting aspect that Felix and Jesse mention: the comparison that the study in question is making (comaring those that barely made it to those that barely didn’t) .
The empirical strategy follows a regression discontinuity (RDD) design in an attempt to mimic a randomized experiment. The idea is that those that barely made it into the school are very similar to those that barely did not make it. The well-known problem is that such a comparison is local in nature, meaning that the estimates not necessarily apply to other pupils that comfortably made it or that were far away from the threshold.
Jesse suggets that the groups close to the cutoff are in a very different position or rank within the school: being top at an average school or being bottom at an elite school does have consequences on its own. Such a negative peer effect may in part explain why the authors don’t find a positive effect of barely making it into an elite school.
What he doesnt mention is that the authors formally test exactly this Big Fish Little Pond Effect. First, they show that Jesse is right: students drop in rank. But it is not as pronounced as one might assume:
The figure shows the rank in the baseline study in Math for the applicants in the 7th grade. Those admitted (just to the right of the red line) drop from the 75th percentile to the median in the O’Bryant school, for example.
The formal test then shows that the interaction between the peer gap (how far a student is from the average baseline score of the target school) and the treatment (being admitted to the elite school) is negative. In less economic jargon, this means that the effect of going to the better school is larger the lower you are in the ranking and decreases the higher you rank. The BFLPE is therefore not very likely to drive the effects of elite schools down.
The evidence therefore does not support Felix’ first concluding statement:
So if your kid doesn’t get in to Stuyvesant, it’s no bad thing — in fact, it might well be a good thing, given that your kid would probably in that case have been near the bottom of the Stuyvesant class.
But it does seem to support his second:
And before you shell out on school fees, ask yourself whether the money wouldn’t be better spent on other forms of education — books, computers, travel, theater, and the like. School’s important. But it’s not nearly as important as most middle-class parents think it is.
Me? I personally thought that the way kids are educated at school does matter, so evidence like this is somewhat discouraging. But maybe it really doesn’t apply to the brightest kids as much and money is better spent on other activities as Felix suggests.