Pages

Thursday, 5 September 2013

Teach First boosts GCSE grades


Today sees the publication of the most comprehensive quantitative analysis of Teach First's impact to date.

The paper - by Dr Becky Allen and Jay Allnut - uses several different methodologies to see whether schools and departments with Teach First teachers achieve better GCSE grades than similar ones that don't. Their two key findings are:

1)    That schools using Teach First see an improvement of around one GCSE grade per pupil across their "best eight" GCSEs. If this doesn't sound like much bear in mind that most secondary schools involved with the programme have only a couple of teachers from Teach First out of a staff of 80-100. Furthermore the researchers only looked at Teach Firsters in their first three years of teaching.

2)    That subject departments containing at least one Teach First teacher see an improvement of about 5% of a grade in that subject when compared to other departments in the same schools. As the authors note given that only 1 in 6 of a department's teachers (on average) will be Teach Firsters this means that if all of that 5% is being generated in their classrooms they will be improving their pupil's results by 30%. Which is a very significant increase. As the authors put it:

"Our estimate of impact of the order of at least 5% of a subject grade could be as high as 30% of a grade if we assume no spillovers of participation to other teachers in the same department. This implies that Teach First participants are highly effective, on average, compared to those they have displaced."

These are really encouraging results but like all good research the paper raises some interesting questions for further study - many of which are noted by the authors in the text. For instance:

1) How much of the impact found is due to Teach First's highly selective recruitment model? The authors think most of it but the effects of recruitment can't easily be separated from the different training model and the sense of mission inculcated during this training. If these things are partly responsible for the impact what could other teacher training programme learn?

2) How much of the departmental impact is down to the Teach Firster's own teaching and how much is it do with the knock-on effects of them joining on the rest of the department? The authors think knock-on effects (or spillover effects as they call them) are important. If this is right then it raises some interesting questions about the validity of value-added models for individual teachers.

3) Are there knock-on effects between departments? So - for instance - does an improvement in the English department also help the Maths department? If it does then the methodology will have underestimated the impact of Teach First teachers (as it compares improvements in departments with Teach Firsters with those without them in the same school).

4) What are the long-term impacts on schools who stay with the programme over many years? Do Teach Firsters who stay in teaching continue to have a strong impact? Do they make particularly effective leaders?

5) What impact will the expansion of Teach First have? This research covers a period when we were recruiting between 200-500 teachers a year. Will be able to maintain that impact when recruiting 2000 - as we plan to do in two years' time? We certainly hope so - and have no intention of consciously dropping the quality bar.




 

4 comments:

  1. This seems convincing evidence Teach First works a bit and some progress towards understanding how it works from Mujis (2010) but as the authors say this analysis cannot disprove some key hypotheses about how it works?
    As they say TF participants may raise school attainment by:

    - being better at teaching
    - Chiding jaded colleagues out of complacency
    - …or represent other things dynamic management are doing that has nothing to do with TF participants

    It immediately seems implausible TF participants directly impact total school attainment by their better teaching as there are only around 4 are in each secondary school (2 per year) out of an average 125 members of staff (DFE). It seems most unlikely four teachers (two rubbish in their first year) can raise the attainment of a school single handed by teaching their pupils better, not least as they simply do not teach enough pupils.

    The paper claims to separate out the individual influence of TF participants alone from the effect of dynamic management by comparing the results of schools participating with TF with a control group of schools that are yet to join but subsequently do. Thus these two groups of schools are supposed to have the same level of managerial dynamism that might be doing other things that explain rises in school results. However, it seems possible that managerial teams that are ‘early adopters’ are a different calibre to ‘followers’ who catch up with trends after it starts to look cool. Also the later adoption of TF might come about because of changes in management personnel/style of governance/academisation impetus. So the one grade in best eight difference in performance between early TF and late adopters might still be caused by longer standing higher managerial chutzpah of the early adopters doing others things nothing to do with TF that cause results to rise.

    Analysis showing that departments with TF participants do better than departments without in the same school, as the authors say, offer a better indication of the impact of TF participants as it allows sight of the effect on colleagues TF work closely with and on colleagues TF do not work closely with. Also all departments are under the same senior management so allows comparison of departments with and without TF participants led by the same general management across the school.
    This analysis seems to allow the possibility that their better teaching raises results of the pupils they teach as they have a chance of impacting results by their teaching alone if they are 1 in 6 teachers. However, this still seems unlikely as many departments are bigger and TF participants being rubbish in their first year leaves the one to teach so much better than their colleagues that the results for the whole departments go up. So, it seems likely these effects are TF participants influencing their colleagues with whom they work closely to raise their game.

    It is commonly assumed main effect of TF will be on raising the attainment of the pupils they teach, evidence from datasets that allow this analysis in the US indicate they do (Xu, Hannaway & Taylor 2011). This study seems to suggest its main effect might be putting the cat among the pigeons in departments and in the staff room. Surely we should test these hypotheses about how it works whilst we are spending so much public money on it.

    It seems inexplicable that the UK is unable to find the data to assess the impact on the main effect of a major initiative after 12 years, £94 million of public money over the last three years and with the UKs much more coherent schools data. Which teacher taught what is in schools MIS if you know where to look.

    As an aside why don’t people write these things for non-specialist readership; how many Educationalists and Teach First senior management have degrees econometrics needed to follow this stuff, and why write this way just before presenting it to a conference on widening the impact of education research?

    ReplyDelete
  2. Also glad your analysis so closely agrees with my own from August
    http://www.consider-ed.org.uk/so-do-we-actually-agree-on-anything/

    ReplyDelete
  3. For those of you applying for Teach First and receiving offers. I write a blog all about Teach First , with posts about the process of getting in, general tips and about my experience so far.

    ReplyDelete
  4. The research paper proofreading is exactly what we need to focus on. Browse online proceedings of recent major conference or journal contents in your research topic area from digital libraries.

    ReplyDelete