Negative peer effects and elite schools

[Warning: this post has a high nerd-factor, read at your own risk]

The recent paper by Angrist et al. „The Elite Illusion: Achievement Effects at Boston and New York Exam Schools“ has received some attention recently, among others by Felix Salmon and Jesse Anttila-Hughes. It shows that attending an elite school does not increase your achievements.

I want to briefly discuss one interesting aspect that Felix and Jesse mention: the comparison that the study in question is making (comaring those that barely made it to those that barely didn’t) .

The empirical strategy follows a regression discontinuity (RDD) design in an attempt to mimic a randomized experiment. The idea is that those that barely made it into the school are very similar to those that barely did not make it. The well-known problem is that such a comparison is local in nature, meaning that the estimates not necessarily apply to other pupils that comfortably made it or that were far away from the threshold.

Jesse suggets that the groups close to the cutoff are in a very different position or rank within the school: being top at an average school or being bottom at an elite school does have consequences on its own. Such a negative peer effect may in part explain why the authors don’t find a positive effect of barely making it into an elite school.

What he doesnt mention is that the authors formally test exactly this Big Fish Little Pond Effect. First, they show that Jesse is right: students drop in rank. But it is not as pronounced as one might assume:

The figure shows the rank in the baseline study in Math for the applicants in the 7th grade. Those admitted (just to the right of the red line) drop from the 75th percentile to the median in the O’Bryant school, for example.

The formal test then shows that the interaction between the peer gap (how far a student is from the average baseline score of the target school) and the treatment (being admitted to the elite school) is negative. In less economic jargon, this means that the effect of going to the better school is larger the lower you are in the ranking and decreases the higher you rank. The BFLPE is therefore not very likely to drive the effects of elite schools down.

The evidence therefore does not support Felix’ first concluding statement:

So if your kid doesn’t get in to Stuyvesant, it’s no bad thing — in fact, it might well be a good thing, given that your kid would probably in that case have been near the bottom of the Stuyvesant class.

But it does seem to support his second:

And before you shell out on school fees, ask yourself whether the money wouldn’t be better spent on other forms of education — books, computers, travel, theater, and the like. School’s important. But it’s not nearly as important as most middle-class parents think it is.

Me? I personally thought that the way kids are educated at school does matter, so evidence like this is somewhat discouraging. But maybe it really doesn’t apply to the brightest kids as much and money is better spent on other activities as Felix suggests.

Kommentare

  1. Jopa schreibt:

    Maybe the main attraction of such schools for ambitious parents is not merely the presumed better education, but more the hope for some early networking anyway?

    • kantoos schreibt:

      @ jopa

      Absolutely, but wouldn’t you assume that grouping very ambitious and smart kids in a school should have some effect on learning outcomes as well? What does that say about the German Gymnasium?

  2. Alex F. schreibt:

    „Motivation by ranking“ is bad in itself. But this is one of the things teacher can’t really teach, because if they do, they contradict themselves. It’s like commanding: “Don’t listen to me!”
    The moment you start something „new“, you will get a lot of resistance. If you underperform because you’re used to be treated as top of the class, you will have a hard time.

    Just take Scott Sumner. Sumner is a brilliant guy, but he constantly belittles himself because of his low ranking. Sumner doesn’t teach at an Elite University, so how dare he to advocate NGDP targeting? This is quite funny for people outside of Academia.

    In the case of Sumner, everything turned out well, Sumner succeeded, NGDP targeting is now mainstream. But I imagine there are a lot of great, but unconventional ideas out there, yet the people who harbor them, don’t have the guts to promote them.
    W.B. Yeats: „The best lack all conviction, while the worst / Are full of passionate intensity“

  3. Martin schreibt:

    @kantoos
    „Absolutely, but wouldn’t you assume that grouping very ambitious and smart kids in a school should have some effect on learning outcomes as well? What does that say about the German Gymnasium?“

    But this study doesn’t test, if there are overall better or worse outcomes, but tests those, who are close to the threshhold. So what does this to the German Gymnasium? First I would be cautious to just translate these results one to one.
    But if you do, it actually kills the best arguments against it. Obviously for kids, that are close to the threshhold, it doesn’t make a big difference for them, if they are admitted or not – as well not because of the formal qualification issues, because to get an „Abitur“ later, if you have a good „Realschulabschluss“ is very easy.
    That the Gymnasium is beneficial for those at the very top is very likely and not related to this study in any way.

  4. Rien Huizer schreibt:

    @Kantoos

    Obviously this is an area that is hard to research. Also I find it hard to get the take away here (apart from the surprising fact that those marginal entrants do quite a bit better than I expected, so it is worth the try if you are a parent, excellent of you to identify that as an aspect of the study). Of course the three schools investigated here are still state schools. There is better quality education (but maybe not better quality co-students) available in the private sector and it would have been nice to include data on that as well. Also, it would have been useful to know the entrant’s dropout rate during their stay at the schools. And of course their later life experiences.

    In Australia there are similar elite state schools. that score almost on par (all students so nothing to do with this „borderline“ study) with the best private schools.However, all -girls schools (by definition private, state schools cannot discriminate) score much better than the best mixed schools (incl the elite state schools with their gifted child-friendly curriculum) or all- boys schools. Maybe the proper research questions are:
    - does testing predict school success in general
    - what tests are the most predictive?
    - are there combinations of specific tests and specific learning environments that give even better outcomes
    - why do (average, but typically from wealthier and better educated parents) children who attend private schools without competitive entrance procedures (you just enter the kid into the waiting list when it is born and make a deposit) do as well or better than very bright, selected kids from the state school system that get a place in one of the elite schools?

    I guess the takeaway should be that the tests/exams in question may predict suitability of the top students (usually you do not need a test for that) and that the borderliners (because they do not do as poorly as one would expect) that do not get a place, just have bad luck.

    I belong to the generation that had to sit a very difficult entrance exam (at the age of 12) to get admission to a „gymnasium“ and had a fist hand view of the usefulness of that test (numbers are from a very distant memory): of the 120 first year students admitted, less than 30% finished with a diploma after 7 years, and only 15% after six. This after rejecting maybe 200 candidates at the gate. What happened to the rejects? They went to a slightly easier school, often made it to university nevertheless, while the drop-outs at the gymnasium tended to have a much more problematic adolescenthood. Of course the school was proud of the rejects and the dropouts – necessary to fill the alumni ranks with professors, doctors and top civil servants. Lessons? Entry exams: probrematic. Deliberately aloof/unpedagogic forms of teaching (as in university) : toughen/weed out? Result at university? typically scholarly, little affinity with practical things, like eg business. Clearly, this was an example of a school with highly selective admission on purely adademic criteria and combined with methods of instruction that are the opposite of modern schools for highly gifted children.

    I guess that is pretty similar to the German and French effects of „classical“ education. Obsolete? Maybe not. Too expensive (rejects, dropouts, excessive bias on scholarship?) Yes, probably. Better alternatives? Probably not (except of course better methods of instruction)…As far as I know, In Belgium they still have a system that is close to the old model, with similar results. But then again, Belgian middle class culture is quite conservative.

Hinterlasse einen Kommentar zu Jopa Antwort abbrechen

Gravatar
WordPress.com-Logo

Melde Dich bitte bei WordPress.com an, um einen Kommentar auf deinem Blog zu schreiben.

Twitter-Bild

Du kommentierst mit Deinem Twitter-Konto. Log Out )

Facebook-Foto

Du kommentierst mit Deinem Facebook-Konto. Log Out )

Verbinde mit %s

Follow

Get every new post delivered to your Inbox.

Join 73 other followers