Pupil Premium accountability

Professor Becky Allen of the UCL Institute of Education has written important articles about the Pupil Premium (PP) that I discuss here and here.

It is perfectly reasonable for OfSTED to demand accountability for additional funding generated by the PP. Professor Allen draws attention to the frequency of critical comments like this in OfSTED reports.

“The leaders and managers do not focus sharply enough on evaluating the amount of progress in learning made by the various groups of pupils at the school, particularly the pupils eligible for the pupil premium …”

My articles support the reservations of Professor Allen about the perverse outcomes that arise from attempts to determine such accountability. However, the determining of accountability remains necessary.

This is my solution to the problem in secondary schools. It is based on relating GCSE attainment to Y7 cognitive ability data for all pupils  including those identified as PP using the current DfE criteria.

1. Screen all intake Y7 pupils with Cognitive Ability Tests (CATs).

This will establish the same general patterns in relation to SATs, CATs and Social and Economic Status (SES) data that John Mountford found in his research that is reported here.

These data have been further summarised by John as follows.

School A B D E F G
% FSM 12% 10% 9% 7% 7% 22%
SATs Y7 mean 105.68 103.45 105.91 106.42 107.07 105.22
CATs Y7 mean 100.26 96.76 105.92 104.76 109.38 97.86
FSM SATs mean 102.41 100.33 104.00 106.05 104.44 98.18
FSM CATs mean 92.91 90.80 98.83 102.13 97.28 89.93
Cohort – FSM SATs  3.27 3.12 1.91 0.37  2.63  7.04
Cohort – FSM CATs  7.35 5.96  7.09  2.63  12.1  7.93

 

School H I J K L
% FSM 44% 15% 15% 4% 15%
SATs Y7 mean 100.68 105.42 105.42 109.14 106.57
CATs Y7 mean 95.58 103.35 103.35 107.61 105.71
FSM SATs mean 100.32 101.83 101.83 104.33 102.37
FSM CATs mean 94.77 92.85 92.85 97.77 97.25
Cohort – FSM SATs  0.36  3.59 3.59 5.11 4.2
Cohort – FSM CATs 0.81  10.5 10.5 9.84 8.46

The data for school C are omitted because of comparability issues.

2. Produce a ‘scatter-chart’ showing the GCSE attainment of each pupil against the Y7 CATs score for that pupil

The following chart appears in a number of my articles illustrating how to validly compare the GCSE attainment data for different schools taking proper account of differences in mean intake cognitive ability.

Graph2

However, instead of the X axis showing the mean intake CATs scores of schools, it should be used for the Y7 CATs score of every individual pupil. The data points of PP pupils on the chart, would be labelled (eg with a different style of data point). The measure chosen for the Y axis could be the DfE defined ‘Attainment 8’, or other measure chosen by the school to recognise more attainment in technical and creative subjects. The school would have to make the case for the chosen measure with OfSTED.

As in the above chart, the regression line (that Excel can produce for you) shows the average performance of pupils in the school in relation to their individual CATs scores. Students appearing above the regression line have done better than the school average and  those appearing below the regression line have done worse.

On the basis of John’s and national data published by the CATs provider GL Assessment (p10), it would be expected that PP pupils would be bunched towards the left hand side of the chart on account of their lower mean cognitive abilities. However any such students that appear on or above the regression line have closed any SES gap between them and their non PP peers. The general pattern for the school will be obvious from the chart.

3. The school should then reflect the outcomes in terms of its curriculum and teaching and learning policies

How should schools best combat social and educational disadvantage? This is addressed in this article.  Becky (Professor Allen) has also pointed to the answers to this question in her articles.

We both agree that a change in DfE policy should result in higher proportions of educationally disadvantaged pupils in school intake cohorts generating enhanced general funding for the school to reflect the increased costs of the effective teaching and learning methods (for all pupils) that are needed. 

Even in the absence of any change in DfE policy, schools can and should use approaches to teaching and learning for all pupils that are proven to be effective. For example, those recommended by EEF for science teaching, that in fact work across the curriculum.

Primary schools can help themselves and their secondary colleagues, by also using cognitively enhancing approaches such as P4C. Further cognitive enhancement can be stimulated by changing school cultures as explained here. Don’t be put off by the title of the article – it really is relevant.

In terms of how the DfE should determine the amount of extra funding to be delegated to schools, together with how to hold schools accountable for outcomes, I believe the answer to both questions was provided by the Cumbria LEA in the early 1990s. I was one of the Cumbria heads that served on the LEA Working Party that devised the approach, so I know in detail how it worked. In those days there were substantial ‘Non-Statutory Special Needs’ funds to be distributed to schools through the funding formula of each LEA. Unlike other LEAs, Cumbria rejected a formula based on FSM in favour of Cognitive Ability Tests (CATs) data obtained from screening all Y7 pupils in October of the intake year. The Cumbria formula then delegated additional funds to schools, not individual pupils, through a formula driven by the numbers of pupils with CATs scores at various threshold levels below 85 (-1SD). Where there were significant differences in the scores on the three sub-tests (Verbal, Non-Verbal and Quantitative Reasoning), the CATs profiles for each pupil should prompt further testing for Specific Learning Difficulties (SLD), so enabling specific intervention for individual students, reflecting Becky’s important point about the diversity of learning needs.

Such charts have further uses. They are an even better way of presenting data for schools to evaluate their own standards and progress than the Cumbria CATs/school GCSE performance scatter-graph. This is because they can provide fine detail information over time for the school. The regression line enables the mean GCSE performance for the school to be calculated for sub-groups of cognitive abilities. For example, schools like Mossbourne Academy have ‘fair banding’ admissions policies, in which there are admission limits for each CAT band. Mossbourne. like many Academies have quartile bands.

A: 110 and above

B: 100-109

C:  90 – 99

D: below 90

The mean GCSE performance of the school for each quartile boundary can be read from the regression line, including the mean school performance for pupils of national average cognitive ability (CATs score = 100)

If in successive years the school becomes more effective in terms of teaching and learning then the school GCSE performance at each quartile boundary will increase (and vice-versa). Finer detail is also available. For example there may be differences in improvement/deterioration between the ability bands.

So GCSE attainment /pupil CATs score scatter-graphs and regression lines can not only provide sound pupil premium accountability, but much else besides for driving school improvement.

Advertisements
This entry was posted in Blogs. Bookmark the permalink.

1 Response to Pupil Premium accountability

  1. John Mountford says:

    Roger, to me this is the most revealing analysis of the SATs/CATs conundrum you have carried out to date. What it marks is a way ahead. It shows us a way to tackle the current universally accepted muddle of school testing and examinations and create a system that offers fairness, value for money and a better deal for children and students of all ages and abilities.

    At last, even Ofsted has responded to a moral imperative to decry the situation whereby schools have elected to game the system rather than face the prospect of failing to meet arbitrary ‘standards’ totally unrelated to their intake profile, as imposed and championed by political parties of every hue. The damage caused by several decades of fluctuating, and too often, harmful political meddling in education is without question. Children’s well being, teacher recruitment and retention and the vast sums wasted in costly and ineffective structural reforms are just a few of the troubling consequences that mark out this dark period.

    It is time to change, despite the likely protestations from the profession that change is the last thing they want.

    What you outline in this piece has the potential to re-shape and improve education outcomes for students. Equally, it creates an opportunity for teachers to regain their professional oversight over what happens in their classrooms for the benefit of the pupils they serve. For this to happen two further developments are necessary and urgent.

    Firstly, teachers’ professional associations must stand up for their members by organising determined opposition to the disastrous political initiatives that have created the problems besetting education. In doing so they will be creating an environment where our young people know that safeguarding their futures is what it is all about.

    Secondly we need, as a society, to take the bold step of removing education governance from the party political sphere. We need a National Commission for the Governance of Education, with non-partisan representation to ensure that reforms are based on sound evidence and the projected future needs of ALL stakeholders.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s