The EEF casts new light on ‘The Attainment Gap’

This article was updated on 30 January 2018 to reflect further important information received.

The Education Endowment Foundation (EEF) has published what is probably the most comprehensive study yet on ‘The Attainment Gap’, which has been the principal concern of ‘The Social Mobility Foundation‘, ‘The Sutton Trust’ and successive incarnations of the Department for Education and its Opposition shadows over the last three decades.

The EEF summarises ‘The Attainment Gap’ as follows.

“The gap in outcomes between those students from the least well-off backgrounds and their classmates is already evident by the time they begin school, aged 5. Over the next 11 years of full-time education, it worsens [to] grow from age 5 to 16:

  • there is a 4.3 month gap at the start of school between disadvantaged children and their classmates;
  • this more than doubles to 9.5 months by the end of primary school; and
  • then more than doubles again, to 19.3 months, by the end of secondary school.”

‘The Attainment Gap’, is based on DfE school performance data at KS1, KS2, and KS4. These data are primarily intended to drive the marketised education system in terms of OfSTED judgements and league tables. This makes them ‘high stakes’ measures for schools, which has implications for their validity.

There is a moral dimension of ‘The Attainment Gap’ arising from those students from the least well-off backgrounds performing worse than their more affluent classmates. This has led to the political right using the issue to condemn comprehensive schools in favour of academic selection and/or extreme regimes of instruction-based teaching backed up by coercive sanction-driven discipline systems.

The fundamental reason for the apparent ‘attainment gap’ is that pupils from poorer backgrounds tend to have lower cognitive abilities. That this fact is unrecognised, ignored or rejected for ideological reasons is why the ‘attainment gap’ has persisted despite all the Labour and Conservative government initiatives, ‘zero tolerance of failure’, head sackings, school closures, Academisations and Free School promotions of recent decades.

None have made any impression because they are remedies for something that does not exist, based on an incorrect diagnosis of the problem.

The key EEF findings are as follows. 

  1. The widest attainment gaps relate to FSM and SEN pupils, but not those that have English as a second language.

While it is often assumed that not having English as the first language must be a significant disadvantage for primary pupils, the facts do not support this. The EEF study is just the latest to come to this conclusion.

Cognitive Ability Test (CATs) data, however, do show a clear link with Socio-Economic Status (SES), for which eligibility for Free School Meals (FSM)  is a reliable proxy. This relationship between poverty and cognitive ability has been known to heads and governors of secondary schools for decades, which is why after the 1988 Education Reform Act compelled all schools to compete on the basis of the raw aggregated attainment of their pupils, those schools that had powers over their admissions policies (Academies and Religious Schools) were increasingly inclined to find ways of cutting down the numbers of FSM, and therefore less cognitively able, children they had to admit.

For many years the Cumbria LEA produced CATs/School Attainment data that showed this relationship, so enabling judgements to be made about the relative effectiveness of its schools that took account of the cognitive ability profiles of admission cohorts. When Labour took control of the county council it banned the production of these performance data. For the whole of its period of office, successive Labour governments regarded any mention of schools’ cognitive ability admissions profiles as being, ‘an unacceptable excuse for failure’. This has remained the view of subsequent Conservative-led governments leading directly to the continuation of ‘The Attainment Gap’ fallacy.

Academisation has taken away the ability of Local Authorities to impose economically and therefore cognitively balanced catchment areas and has brought about an admissions free for all. This has resulted in those schools located in affluent post codes (therefore admitting fewer less bright children on the basis of proximity) getting better aggregated exam results, climbing the local league tables, having more Y7 applications and therefore being able to deny entry to ever greater numbers of poorer pupils living further from the school, so increasing the mean intake cognitive ability, therefore getting better exam results etc. etc.

The converse has happened to schools located in the heart of poor communities. This admission inequality issue has been successfully addressed by the introduction of CATs driven fair banding, which while being widely adopted by Academies, has not been available to LA schools, except in the London Borough of Hackney. I investigated this in detail using real CATs data in my study of Mossbourne Academy and the Hackney Fair Banding admissions system. This appears as Part 4 of my book, ‘Learning Matters’.

That school students from less affluent postcodes achieve less well than their more affluent classmates is therefore explained by the associated differences in mean cognitive ability, which also accounts for the generally lower attainment of students with various categories of SEN.

  1. The gap persists in all types of secondary schools

Attainment 8 scores for all pupils is higher in ‘Outstanding’- or ’Good’-rated schools, than (on average) in schools rated as either ‘Requires improvement’ or ‘Inadequate’. However, the size of the Attainment 8 gap between disadvantaged and non-disadvantaged pupils is all but identical across all four Ofsted-rated categories of school. It is not, as might be expected, a problem that predominates in schools classified as under-performing: it is found to a similar degree in all types of schools. An EEF chart shows the GCSE outcomes (Attainment 8 scores) for disadvantaged pupils and non-disadvantaged pupils grouped according to their school’s overall effectiveness, as assessed by Ofsted.

‘Outstanding’ schools are no more effective than ‘Special Measures’ schools in closing ‘The Attainment Gap’.

The entire focus of government education policy has been directed onto increasing the numbers of pupils in ‘good’ or ‘outstanding’ schools. At successive Prime Ministers Questions, regardless of what Jeremy Corbyn actually asks, Theresa May never fails to mention the success of post 2010 Conservative governments in increasing these numbers.

Yet the EEF finds that this has had no effect at all on reducing ‘The Attainment Gap’. This is a devastating fact that calls into question the entire rationale of both government education policy and the relevance of OfSTED in improving the attainment of lower attaining students.

Schools have been recently receiving their Inspection Data Summary Reports (IDSRs) from OfSTED. This is the latest national framework for judging school performance and effectiveness. It has replaced ‘RAISEonline’ and before that the PANDAs that I was familiar with as a headteacher in the 1990s. The methodology is to attempt to judge the attainment and progress of pupils in KS1, KS2 and KS4 in each of four groups. The first three of these relate to Low, Middle and High attainers in the previous Key Stage (or in the Reception Year for KS1). But the fourth category is ‘Disadvantaged’ pupils, which is those eligible for FSM.

The OfSTED judgements for all schools will therefore depend on the progress made by those pupils in the school judged to be ‘disadvantaged’ by their eligibility for FSM. The IDSA goes further by identifying such pupils in ‘scatterplots’, the implication being that their individual progress or lack of progress will count for or against the school’s overall OfSTED judgement. However such pupils may well be performing in accordance with, or better than their CATs scores and still be judged to be ‘underperforming’ on the false assumption that they have average cognitive abilities when these are in fact lower. Without CATs data is impossible to determine any independent effect of eligibility for FSM on attainment.

However, if there is indeed no ‘attainment gap’, and all school students perform, on average, in accordance with their cognitive ability as determined by CATs, then a lower performance by the ‘disadvantaged’ group is exactly what should be expected. This will result in schools being penalised by OfSTED just for having greater proportions of FSM pupils in the school. Despite constant denials, this has always been the general pattern of OfSTED school judgements.

This is exactly what has happened. The ‘left establishment’ blogsite ‘Reclaiming Schools’ published this article on 30 January 2018.

“Schools in North East England are under attack again. According to Progress 8 scores, its schools are the least effective in the country, with the highest percentage coming ‘below the floor’. But Progress 8 is a flawed and misleading measure. It assumes that social factors make no difference. Once again, the Government are in denial about poverty and the economy. It’s so much easier to attack teachers again.”

This reaction is entirely predicable. The political left prefers to interpret ‘The Attainment Gap’ as a form of prejudicial class-based discrimination further confirming the malign outcomes of growing social and economic inequality. The ‘Reclaiming Schools’ schools article goes on as follows.

“Poverty has a big impact on pupils’ progress: on average, students on free school meals score -0.5 on Progress 8. (-0.5 is also the threshold for ‘below the floor’.) Schools with large numbers of FSM students are far more likely to score below. In the North East, 17% of students are FSM (13% nationally). In some places it’s worse:

  • 24% Middlesbrough
  • 23% Newcastle
  • 20% Sunderland
  • 19% South Tyneside
  • 19% Hartlepool.

Not surprisingly, all these areas have large numbers of schools ‘below the floor’.”

It is not ‘poverty’ that is the cause of the lower attainment, but the lower mean cognitive ability that prevails in these impoverished communities. I can see why this truth is difficult to accept for Labour politicians and the trade unions, but the truth it is. We know this from ‘GL Assessment’, the commercial marketer of the national Cognitive Ability Tests (CATs) formerly provided by NfER. P10 of this 2009/10 report provides the irrefutable data.

The 146,075 students without FSM obtained a mean combined score of 102 (55th percentile)

The 27,536 FSM students obtained obtained a mean combined score of 92 (30th percentile)

The purpose of the Report is given in its introduction.

GL Assessment provides updated tables and progress charts to enable schools to estimate
pupils’ GCSE subject grades based on their CAT scores. This report explains how to use these tables and charts. It also gives guidance on setting targets and discussing them with individual pupils. Information on how the estimates (or indicators) were developed, and on how to calculate estimates for groups, is included in the appendix. The results of a recent large scale study looking at the relationship between CAT scores and pupil/school factors is included at the end of the appendix [p10] for your information.

It could not be clearer. The attainment of FSM students would be predicted to be far below their more affluent classmates. See the charts for maths in Figure 1 of p2. These predicted that while 75% of non-FSM students were predicted to obtain a C+ grade, only 28% FSM students would be so predicted. Similar extreme differences can be found in other predictions of attainment.

Where the ‘Reclaiming Schools’ article is correct, is that the schools in the North East are indeed being unfairly targeted. On the basis of the GL Assessment predictions, the schools would appear to have obtained far better GCSE results for their FSM students than their CATs scores would have predicted.

I am the former head of an inner-urban school in Barrow-in-Furness, where the mean intake CATs score was 85, but the astonishing progress of our students was never recognised as unsurprisingly the aggregated GCSE results never met the ‘floor targets’ of the time. No head or Labour politician could ever argue locally that the reason lay with the low mean Cognitive Ability of the admission cohorts. The eventual ‘solution’ was the closure and demolition of all three mainland non-religious schools in the town and their replacement by an £multi-million Academy scheme that proved to be a massive failure from its opening, leading to hundreds of Barrow pupils now travelling by bus and train every day to attend the LA schools in the nearby towns of Dalton and Ulverston.

All of this is confirmed by a ‘Schools Week’ report that concluded that schools with lower FSM numbers were more likely to be judged ‘outstanding’ and/or to improve their OfSTED grades.

As the concerns  from both ends of the political spectrum play into their own deeply rooted belief systems, there is little political appetite in any of the main political Parties or the unions for questioning whether the  ‘Attainment Gap’ actually exists.

  1. Regional variations in ‘disadvantaged’ pupil attainment

The EEF reports as follows.

Looking at the attainment gap on a regional basis, it is the performance of pupils in London which stands out. [The EEF tables] show that a majority (51%) of London’s FSM-eligible pupils achieved A*-C grades in GCSE English and maths in 2016. In the neighbouring South East barely one-third of FSM-eligible pupils did so. London’s attainment gap was 19 percentage points; the South East’s was 34 percentage points. FSM-eligible pupils in London were 52% more likely to get 5 or more good GCSEs in 2015 than FSM-eligible pupils in other parts of the country.

The reasons for the transformational improvements in pupils’ outcomes in London in the past 15 years have been much debated. It is not possible to identify for sure why it happened – the causal mechanisms – as the reforms introduced were not robustly evaluated. Researchers have proposed a number of plausible explanations for what has been termed ‘the London effect’, notably: improvements at primary schools from the late 1990s; the London Challenge and other initiatives within secondary schools; and a significant influx of pupils from high-attaining immigrant families.

The last line is significant. While it likely that ‘high attaining immigrant families’ are likely to be a factor, such high attainment matches the higher than national average mean cognitive ability scores of some ethnic groups that are strongly represented in London schools.

This is especially relevant to London, not just because of the high proportion of children from immigrant parents, but also because so many schools take the CATs tests provided by GL Assessment, enabling cognitive ability data to be taken into account by the schools.

The London data are therefore critical to my argument, because it is established that although many immigrant communities remain over-represented in all manner of negative social data, the children nevertheless perform at school according to their (sometimes higher) cognitive ability, not their (often lower) parental affluence.

This effect helps account for the success of so many of the Hackney secondary schools, including Mossbourne Academy, as described and explained in my book, ‘Learning Matters’. My research on matching CATs scores to the GCSE performance of individual students in my own Cumbria headship school showed that there was no significant link between eligibility for FSM and attainment for students with similar CATs scores.

Mossbourne Academy and other Hackney schools have in the past been praised by OfSTED for the high attainment of ‘disadvantaged’ pupils. However the ‘fair banding’ admissions system in effect ‘reserves’ spaces in the high CATs score admission bands for FSM students with high CATs scores. Such students perform well (in accordance with the CATs score) despite their FSM ‘disadvantage’. High FSM postcodes, while being characterised by lower mean CATs scores, still contain some pupils with high CATs scores and ‘fair banding’ admissions policies can find and accept them.

Regrettably the EEF has not included CATs data in its analyses, so it is unable to test my assertions. If it revisits its work, I am confident that it too will conclude that ‘The Attainment Gap’ is an illusion arising from the failure to recognise that FSM eligibility is, on average, not just a proxy for Social Economic Status (SES), but also cognitive ability.

The data needed should be readily available from the Hackney LA, other Multi Academy Trusts (MATs) that use ‘Fair Banding’ admission policies, as well as from GL Assessment.

  1. How can the ‘Attainment Gap’ be successfully addressed?

Clearly, it does not need to be reduced if it does not actually exist. However, the ‘EEF Toolkit’ does contain important evidence for improving the levels of understanding of all school students through effective teaching and learning approaches. The problem with this is that the DfE and many Academy and Free School MATs do not seem to take any notice of the EEF toolkit conclusions, preferring their own evidence-light marketisation ideology-based approaches.

The true pattern is that cognitively developmental approaches are the most effective. The educational developmental trajectory of school students is not fixed by their CATs scores or anything else. This is demonstrated from the five reports published by EEF in July 2017.

Two approaches that work are as follows.

Dialogic Teaching

“Dialogic Teaching aims to improve pupil engagement and attainment by improving the quality of classroom talk. Teachers are trained in strategies that enable pupils to reason, discuss, argue and explain rather than merely respond, in order to develop higher order thinking and articulacy. The programme uses video review, print materials and in-school mentoring to support teachers’ practice across English, maths and science lessons.

This trial found consistent, positive effects in English, science and maths for all children in Year 5, equivalent to about 2 months additional progress.”

This is consistent with other EEF trials focusing on cognitively challenging talk, such as ‘Philosophy for Children’, and ‘Thinking, Doing, Talking Science’. The consistent results across subjects and the lack of any subject specific content in the training suggest that the approach may improve children’s overall thinking and learning skills rather than their knowledge in a given topic.”

This EEF conclusion is important as it recognises that the successful approaches are not based on the learning of factual content but on stimulating and supporting the development of general cognitive ability. Put simply, the pupils made more progress because the teaching and learning methods used made them cleverer. It is important to note that pupil’s confidence and performance improved in all subjects, not just the ones directly relevant to what the ‘classroom talk’ was about. This is the claim of the long-standing ‘cognitive acceleration‘ movement led by Michael Shayer and the late Philip Adey, backed by a huge amount of peer reviewed research.

Thinking, Doing, Talking Science

“Thinking, Doing, Talking Science (TDTS) is a programme that aims to make science lessons in primary schools more practical, creative and challenging. Teachers are trained in a repertoire of strategies that aim to encourage pupils to use higher order thinking skills. For example, pupils are posed ‘Big Questions’, such as ‘How do you know that the earth is a sphere?’ that are used to stimulate discussion about scientific topics and the principles of scientific enquiry.”

Thinking, Doing, Talking Science appeared to have a positive impact on the attainment of pupils in science. Overall, Year 5 pupils in schools using the approach made approximately three additional months’ progress.”

This too was an initiative based not on ‘telling by the teacher and listening by the pupils’, but on the development of general cognitive ability through metacognition, pupil/pupil and pupil/teacher talk.

These are followed by three approaches that are not based on cognitive development and are much less effective .

The EEF and the Sutton Trust could have an important role in changing the ruling pedagogy of the English school system for the better, but they first need a rethink about the true nature and actual pedagogic implications of ‘The Attainment Gap’

I welcome comment and debate in response to this article.

Advertisements
This entry was posted in Blogs, Uncategorized. Bookmark the permalink.

3 Responses to The EEF casts new light on ‘The Attainment Gap’

  1. geoffjordan says:

    This is obviously a very,very important issue. With the greatest respect, as someone with no knowledge of the UK educational system these days, I found it difficult to follow and I’m still not clear what the data about cognitive abilities show.

    You say “the fundamental reason for the apparent ‘attainment gap’ is that pupils from poorer backgrounds tend to have lower cognitive abilities”. Yet the London data seem to contradict this. So do pupils from poorer backgrounds have lower cognitive abilities or not? Sorry if I’m just being dull.

    Like

    • Geoff – As EEF argues, the reasons for the ‘London effect’ are probably varied and complex. ‘The London Challenge’ however is probably a factor. This relates to a major programme of co-operation between London schools that led to them adopting a variety of London-wide approaches to teaching and learning that were different to the ‘national initiatives’ implemented elsewhere under the central direction of various incarnations of the Department for Education.

      London also has a very diverse population that includes children of parents of many different ethnic origins. There are significant differences in mean CATs test scores between these groups, with some scoring much higher than the majority population. This means that the children of poorer families from some of these groups have higher mean CATs scores than those of more affluent families in the majority population. The most extreme example is the children of Chinese and other East Asian parents. I speculate about possible cultural and other explanations for this here.

      https://rogertitcombelearningmatters.wordpress.com/2016/11/25/why-are-chinese-children-such-high-achievers-is-it-a-matter-of-wen-and-wu/

      Like

  2. John Mountford says:

    Hi Roger
    Another insightful commentary about what must surely lie at the heart of the debate we should be having about education. Congratulations on your tenacity in keeping the flame alight in this vital debate and on the clarity of the argument presented.

    The section, ‘ The attainment gap: its impact on children and young people’, opens with the following remarks,

    “A good education, with the qualifications to show for it, can transform lives for
    the better.
    Conversely, young people who finish their studies without attaining the expected
    standards will struggle both in further study and the world of work.”

    Though this is an obvious conclusion to be drawn about the transformative impact of a good education, it leaves us with a pressing question. What is a good education? Maybe I am a little sceptical of EEF in seeking such a clarification. After all, it has made a huge contribution in research into effective teaching and learning. However, the debate about the fundamental question about what education is for has yet to take place.

    I would suggest this report largely plays to the idea that the purpose of education is primarily about the qualifications young people have to show at the end of the process. As important as this connection certainly is, I question that it will ever be enough. As all your writing indicates, Roger, education is more than the sum of its parts and the question of intent is so important. I am concerned that the views expressed in this report support the prevailing mentality in politics, the media, in the public arena and even among the professionals, that education can translate so reliably into the analysis of statistics and trends. This is not and cannot be enough.

    In response to your main argument, the report’s reference to the attainment of “expected standards” is moot. What are these expectations based upon? The report ploughs the same furrow as so many others have done in failing to connect the relevance of cognitive capacity and its clear relevance to the outcomes for individual students.

    I agree wholeheartedly with your comment, “pupils may well be performing in accordance with, or better than their CATs scores and still be judged to be ‘underperforming’ on the false assumption that they have average cognitive abilities when these are in fact lower.”

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s