Pages

Saturday 28 June 2014

The London Schools Effect - what have we learned this week?



Perhaps the biggest question in education policy over the past few years is why the outcomes for London schools have been improving so much faster than in the rest of the country. I wrote about it here last year. Until now there's been little in the way of research into the question but last week two reports came out - one by the IFS and one from CFBT - that seek to provide some answers.

They both agree that the change in GCSE results has been spectacular. There's plenty of data in both reports on this but I found this graph from the IFS particularly powerful because it relates to a metric that isn't something schools are held accountable to - and so feels like authentic proof that something extraordinary has happened in London.




But what, exactly, has happened? Here the two reports seem to disagree. According to the IFS - whose analysis is purely quantitative the main reasons are:
  • Changes in pupil and school characteristics - in particular London and other inner-city areas have seen an increase in pupils from a range of ethnic backgrounds (partly) as a result of immigration. The IFS analysis suggests this accounts for about half the improvement in London between 2002-2012.
  • Changes in "prior attainment" - the authors argue that once higher levels of attainment in key stage 2 (end of primary) tests are taken into account then the "London effect" in secondaries looks less impressive. Indeed once prior attainment and changes in pupil/school characteristics have been controlled for the gap between London and the rest of the country falls from 21 percentage points in the  5 A*-C GCSE with English and Maths measure to just 5 percentage points. Moreover this gap is fairly stable between 2002-2012 - though it does increase a by about 2 percentage points over the period.
  • There was a big increase in key stage 2 schools for disadvantaged pupils between 1999-2003 and that led to big increases in GCSE scores for these pupils between 2004-08 - but the GCSE improvement was actually the result of prior attainment. The authors hypothesise this may be due to the introduction of "national strategies" in primary literacy and numeracy in the late 90s - these were piloted in inner London authorities (as well as some other urban areas e.g. Liverpool).
  • London secondaries do have a better record at getting disadvantaged pupils to stay in education post-16. After controlling for pupil/school characteristics they are around 10 percentage points more likely to stay in education.

The CFBT report does include quantitative analysis but is much more focus on qualitative research - specifically interviews with headteachers, academics, civil servants and other experts. This report argues the key reasons for London's improvement are:
  • Four key "improvement interventions" between 2002 and 2014 - the "London Challenge" (a Labour initiative that used data to focus attention on weaker schools and used better schools to support their improvement); Teach First; the introduction of sponsored academies; and improvements driven by local authorities.
  • They conclude that: "each of these interventions played a significant role in driving improvement. Evaluations of each of these interventions have overall been positive, although the absence of RCT evidence makes it impossible to identify the precise gains from each set of activities. The exact causal mix also varied from borough to borough because there were variations in the level of involvement in London Challenge, variations in the effectiveness of local authority activity, variations in the level of ‘academisation’ and variations in the level of input from Teach First."
  • The authors argue that there were cross-cutting themes covering these interventions and the wider improvement story. In particular - the better use of data; practitioner-led professional development and, particularly, leadership - both politically and at school level.

At first glance it's hard to reconcile the positions taken in the two reports. The IFS focus on primary, and to a lesser extent pupil characteristics, while CFBT focus on secondary policy changes. I think, though, they are two different bits of an extremely complicated jigsaw that hasn't been finished yet - and because of the lack of evidence/data - never will be. Like the apocryphal blind men with the elephant they're looking at different parts of the whole.

1) Both reports probably underestimate the importance of changes in pupil characteristics. CFBT completely dismiss this as a driver based on an inadequate analysis of ethnicity data. The IFS analysis is more comprehensive and so does pick up a significant effect but may still miss the true extent because of the limitations of available data on ethnicity. I think this may explain the extent of the "primary effect" in the IFS report. Essentially they're saying the big improvements in GCSE results are partially illusory because they were already built into those pupils' primary attainment. However, they are unable (because of a lack of data) to analyse whether those primary results were also partly illusory because those pupils started primary at a higher level.

There is a clue that this may be a factor in their analysis of Key Stage 1 data for more recent years. Controlling for prior attainment at KS1 reduces the "London effect" at Key Stage 2 by about half. But the authors are unable to do this analysis for the crucial 1999-2003 period when results really improved. They are also unable to look from the beginning of primary - because we don't have baseline assessments when pupils start school.

2) The IFS report probably underestimates the secondary effect. As Chris Cook has shown the London secondary effect at least doubles if you exclude equivalents.

3) The CFBT report definitely underestimates the primary effect because it doesn't look for it. Thought there are some quotes from people who worked in local authorities during the crucial period who highlight their focus on literacy and numeracy during the late 90s.

So pupil characteristics; primary schools and secondary schools all seem to have played a role in boosting attainment in London. The CFBT report is convincing on some of the factors at play in secondaries; the IFS report is convincing that primaries also played some kind of a role. The big questions for me after digesting both reports:

  • Are there "London specific" pupil characteristics that wouldn't be apparent from the available data. E.g. are immigrants who go to London different to those who don't? Are some of the ethnicity effects stronger than indentified because key groups (e.g. Polish) are hidden in larger categories?
  • Are there policy reasons why London primaries improved faster than those elsewhere in the crucial 1999-2003 period? I struggle to buy the idea that the national strategies were the key driver here as they were rolled out nationally (albeit that the pilots were focused on inner London). But the quotes in the CFBT report suggest their might be something here around a general focus on literacy/numeracy. This is a key area for further research.
  • To what extent were the policy interventions (London Challenge, academies etc...) the main reasons for secondary improvement? Or was it more to do with the number of good school leaders during that period? One of the most interesting tables in the CFBT report - pasted below - shows that inner London is the only part of the country where headteacher recruitment has got easier in the last ten year. And the importance of leadership shines through in the interviews conducted for the CFBT report. Is it possible to more closely identify the relationship between individual leaders and school improvement? What can we learn from these leaders?

























And of course the really big question - is any of this replicable in other areas? We're starting to see a raft of local improvement initiatives across the country - Wales Challenge; Somerset Challenge; North East Challenge and so on. It's really important that in these areas we do a better job of evaluating all the interventions put in place from the start so that if we see big improvements we have a better understand of the causes.


Further reading:

The IFS report

The CFBT report

Chris Cook's analysis

Loic Menzies - one of the CFBT authors - on the two reports

The London Challenge evaluation by Merryn Hutchings and others

Transforming Education For All: The Tower Hamlets Story by Chris Husbands et al


23 comments:

  1. I have a few thoughts on this from my own (anecdotal) experience- one is about the geography of London local authorities. When I worked at a school in a large country we were pretty much left to out own devices by the local authority as we were miles from the consultants' base. It was a shock when I started teaching in London to find that all staff were on first name terms with LA staff; they knew our schools well and we knew them. With just a quick phonecall we could invite them in for support. When I became a local authority consultant that was my role too. If a school requested some support or had a concern I could be there in a matter of minutes. We were able to nip problems in the bud. It also meant that we knew every school's results, attainment and progress and could offer interventions such as ISP in a very accurate way. There are also huge numbers of opportunities for London schools that support the curriculum. As a teacher I was always being offered free workshops, visits from specialists, opportunities to take part in projects and activities and competitions, all of which support the broad and balanced curriculum. If we were studying democracy we'd jump on free public transport and visit City Hall or studying Shakespeare go and watch a free play. Although these are anecdotal examples I believe that they go towards supporting a wider and often more aspirational educational offer.

    ReplyDelete
  2. HTSolution là một công ty chuyên cung cấp dịch vụ thiết kế website, gia công phần mềm. Với mong muốn mang đến những sản phẩm chất lượng chuyên nghiệp đến với khách hàng. Chúng tôi không bao giờ mang đến những website kém chất lượng bởi chúng tôi biết tầm quan trọng của website đối với các doanh nghiệp ban. Hãy để chúng tôi trở thành một trong những người bạn cùng chia sẽ những nổi lo với các bạn. Hãy tiên tưởng ở chúng tôi.
    phần mềm quản lý nhà hàng HTRM
    thiết kế website chuyên nghiệp
    phần mềm quản lý quán cà phê
    thiết kế website chất lượng giá rẻ trên toàn quốc
    Dịch vụ phần mềm và thiết kế website
    thiết kế website chất lượng cao tại tphcm
    chỉnh sửa website theo yêu cầu
    thiết kế website bán vé máy online trên toàn quốc
    thiết kế website nhà hàng tiệc cưới tại tphcm
    Hãy liên hệ trực tiếp với chúng tôi để chúng tôi mang đến sự tiện lợi chất lượng tốt nhất cho bạn.

    ReplyDelete
  3. Thanks for this post. I personally prefer to those student accommodations that are near to universities or in major areas of the city. With the help of letting agents it is not difficult to get accommodation according to your standard.


    Closest student accommodation to Manchester University
    All inclusive student accommodation in Manchester
    Vita First Street Manchester

    ReplyDelete

  4. File explorer in windows 10 is much similar that previous versions of microsoft windows if here you get any problem in accessing file explorer then visit our website get help with file explorer in windows 10 for more knowledge.

    ReplyDelete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Education is the key to success. How to paraphrase an article is the basic skill that the student has to have.

    ReplyDelete
  7. Thanks for this post. I personally prefer to those student accommodations that are near to universities or in major areas of the city. With the help of letting agents it is not difficult to get accommodation according to your standard. Good work keep it up.

    Maths tutor London

    ReplyDelete
  8. Excellent post, keep them coming!
    Student living has gone a long way in the last couple of years, and private accommodation providers are really stepping up their game. Would be interesting to see if campus housing is able to keep up.
    student accommodation

    ReplyDelete
  9. Interesting chapter to discuss briefly.Your work is good and knowledgeable..exam solutions

    ReplyDelete
  10. The blog is well organized with content as per readers interest. This is making it to increase blog visitors. Valuable source of data. Looking for more updates.
    Thanks
    Online affiliate programs

    ReplyDelete
  11. If you're Teach First reading this hope you're having a good Saturday

    ReplyDelete
  12. Check out our MASSIVE guide on the best acoustic guitars for beginners. how to play acoustic guitar for beginners

    ReplyDelete
  13. However, in return for the convenience of halls, you may find yourself paying more than you would in a private house or flat.

    All inclusive student accommodation in Chester

    ReplyDelete
  14. Super blog and your presenting style about the Maths tutor London that's are very appreciated good work.

    Math Tutors

    ReplyDelete
  15. techxiphoid very nice article I am subscribing to your blog

    ReplyDelete
  16. jwinstadeliver very nice article I am subscribing to your blog

    ReplyDelete
  17. this is a very informative blog it really helped me and I have shared this on social media http://www.joinmetonight.com

    ReplyDelete
  18. href="http://www.joinmetonight.com">

    ReplyDelete
  19. Loved your blog as it was well researched and offered a knowledgeable resource. You can now acquire customized Dissertation Writing Service from certified experts who promise to offer you a one of kind academic assistance. Embrace yourself to acquire high quality assistance from the most reliable professionals in the block.

    ReplyDelete
  20. This comment has been removed by the author.

    ReplyDelete
  21. It is now time for the BSEB Matric Blueprint 2023 to be released. We provide the most up-to-date information about BSEB. The BSEB 10th Blueprint 2023 PDF has already been issued by the BSEB. On our website, we provide a direct access to the Bihar Board 10th Blueprint 2023. Patna Board 10th Blueprint 2023 In the month of July, the Bihar Board of Secondary Education held SSC Public Exams. Students who have taken the BSEB matric examinations can access the most recent information on our website.

    ReplyDelete
  22. "The vibrant and diverse metropolis, London, is a global cultural hub. 2247 2Nordvpn Change Location Unlocking A World Of Possibilities Its rich history, iconic landmarks, and multicultural atmosphere make it a truly unique city in universe.

    ReplyDelete