It is perfectly reasonable for OfSTED to demand accountability for additional funding generated by the PP. Professor Allen draws attention to the frequency of critical comments like this in OfSTED reports.
“The leaders and managers do not focus sharply enough on evaluating the amount of progress in learning made by the various groups of pupils at the school, particularly the pupils eligible for the pupil premium …”
My articles support the reservations of Professor Allen about the perverse outcomes that arise from attempts to determine such accountability. However, the determining of accountability remains necessary.
This is my solution to the problem in secondary schools. It is based on relating GCSE attainment to Y7 cognitive ability data for all pupils including those identified as PP using the current DfE criteria.
1. Screen all intake Y7 pupils with Cognitive Ability Tests (CATs).
This will establish the same general patterns in relation to SATs, CATs and Social and Economic Status (SES) data that John Mountford found in his research that is reported here.
These data have been further summarised by John as follows.
|SATs Y7 mean||105.68||103.45||105.91||106.42||107.07||105.22|
|CATs Y7 mean||100.26||96.76||105.92||104.76||109.38||97.86|
|FSM SATs mean||102.41||100.33||104.00||106.05||104.44||98.18|
|FSM CATs mean||92.91||90.80||98.83||102.13||97.28||89.93|
|Cohort – FSM SATs||3.27||3.12||1.91||0.37||2.63||7.04|
|Cohort – FSM CATs||7.35||5.96||7.09||2.63||12.1||7.93|
|SATs Y7 mean||100.68||105.42||105.42||109.14||106.57|
|CATs Y7 mean||95.58||103.35||103.35||107.61||105.71|
|FSM SATs mean||100.32||101.83||101.83||104.33||102.37|
|FSM CATs mean||94.77||92.85||92.85||97.77||97.25|
|Cohort – FSM SATs||0.36||3.59||3.59||5.11||4.2|
|Cohort – FSM CATs||0.81||10.5||10.5||9.84||8.46|
The data for school C are omitted because of comparability issues.
2. Produce a ‘scatter-chart’ showing the GCSE attainment of each pupil against the Y7 CATs score for that pupil
The following chart appears in a number of my articles illustrating how to validly compare the GCSE attainment data for different schools taking proper account of differences in mean intake cognitive ability.
However, instead of the X axis showing the mean intake CATs scores of schools, it should be used for the Y7 CATs score of every individual pupil. The data points of PP pupils on the chart, would be labelled (eg with a different style of data point). The measure chosen for the Y axis could be the DfE defined ‘Attainment 8’, or other measure chosen by the school to recognise more attainment in technical and creative subjects. The school would have to make the case for the chosen measure with OfSTED.
As in the above chart, the regression line (that Excel can produce for you) shows the average performance of pupils in the school in relation to their individual CATs scores. Students appearing above the regression line have done better than the school average and those appearing below the regression line have done worse.
On the basis of John’s and national data published by the CATs provider GL Assessment (p10), it would be expected that PP pupils would be bunched towards the left hand side of the chart on account of their lower mean cognitive abilities. However any such students that appear on or above the regression line have closed any SES gap between them and their non PP peers. The general pattern for the school will be obvious from the chart.
3. The school should then reflect the outcomes in terms of its curriculum and teaching and learning policies
We both agree that a change in DfE policy should result in higher proportions of educationally disadvantaged pupils in school intake cohorts generating enhanced general funding for the school to reflect the increased costs of the effective teaching and learning methods (for all pupils) that are needed.
Even in the absence of any change in DfE policy, schools can and should use approaches to teaching and learning for all pupils that are proven to be effective. For example, those recommended by EEF for science teaching, that in fact work across the curriculum.
Primary schools can help themselves and their secondary colleagues, by also using cognitively enhancing approaches such as P4C. Further cognitive enhancement can be stimulated by changing school cultures as explained here. Don’t be put off by the title of the article – it really is relevant.
In terms of how the DfE should determine the amount of extra funding to be delegated to schools, together with how to hold schools accountable for outcomes, I believe the answer to both questions was provided by the Cumbria LEA in the early 1990s. I was one of the Cumbria heads that served on the LEA Working Party that devised the approach, so I know in detail how it worked. In those days there were substantial ‘Non-Statutory Special Needs’ funds to be distributed to schools through the funding formula of each LEA. Unlike other LEAs, Cumbria rejected a formula based on FSM in favour of Cognitive Ability Tests (CATs) data obtained from screening all Y7 pupils in October of the intake year. The Cumbria formula then delegated additional funds to schools, not individual pupils, through a formula driven by the numbers of pupils with CATs scores at various threshold levels below 85 (-1SD). Where there were significant differences in the scores on the three sub-tests (Verbal, Non-Verbal and Quantitative Reasoning), the CATs profiles for each pupil should prompt further testing for Specific Learning Difficulties (SLD), so enabling specific intervention for individual students, reflecting Becky’s important point about the diversity of learning needs.
Such charts have further uses. They are an even better way of presenting data for schools to evaluate their own standards and progress than the Cumbria CATs/school GCSE performance scatter-graph. This is because they can provide fine detail information over time for the school. The regression line enables the mean GCSE performance for the school to be calculated for sub-groups of cognitive abilities. For example, schools like Mossbourne Academy have ‘fair banding’ admissions policies, in which there are admission limits for each CAT band. Mossbourne. like many Academies have quartile bands.
A: 110 and above
C: 90 – 99
D: below 90
The mean GCSE performance of the school for each quartile boundary can be read from the regression line, including the mean school performance for pupils of national average cognitive ability (CATs score = 100)
If in successive years the school becomes more effective in terms of teaching and learning then the school GCSE performance at each quartile boundary will increase (and vice-versa). Finer detail is also available. For example there may be differences in improvement/deterioration between the ability bands.
So GCSE attainment /pupil CATs score scatter-graphs and regression lines can not only provide sound pupil premium accountability, but much else besides for driving school improvement.