Since I published this article I have received a lot of email correspondence from internationally respected academics. The general thrust of this has been to confirm the general validity of my approach. However Professor Richard Lynn informed me that my national IQ data were out of date and provided me with the update published in the appendix of his book, ‘ Lynn & Vanhanen Intelligence: A Unifying construct for the Social Sciences,’
https://www.amazon.co.uk/d/Books/Intelligence-Unifying-Construct-Social-…
I have to emphasise that I am using these IQ data for the purpose of interpreting the 2015 international PISA test results. Others may use the data for other purposes and come to conclusions that I do not support. However, Lynn’s IQ data now come fully referenced as to sources and include updates resulting from the Flynn effect, along with confirmation that the Flynn increase in IQs over time has ceased for pupils in the UK, as noted in Section 5.10 of, ‘Learning Matters’ where I explain this in terms of the degradation of the English education system caused by marketisation.
Most usefully the updated IQ data are given for various ages. In my corrections I either use the median values or those not above but as close to age 15 as possible. This removes the weakness in my previous analysis of using adult IQs as a proxy for the average national IQs of the students taking the PISA tests.
I fully understand that the details of my analysis depend on the quality of the IQ data. This is not uniform, as Lynn makes clear that the evidence base for some countries is much more limited than for others. I note that Lynn has been criticised for underestimating black African IQs. I can easily see why many educationists find the whole IQ field such dangerous territory that they keep well clear. However in relation to my analysis it should be noted that for all countries if IQs are revised upwards then the effectiveness of the education system is revised downwards.
Lynn’s IQ data comes with this foreword.
These IQ have been obtained from the administration of tests of intelligence and the IQs have been calculated in relation to a British mean of 100 and standard deviation of 15. All IQs have been adjusted for Flynn effects, i.e. secular increases in IQ. Flynn effect adjustments up to the year 1980 are 3 IQ points per decade (Flynn, 1987) for all tests except the Progressive Matrices, for which they are 2 IQ points per decade reported for Britain by Lynn and Hampson (1986). The same adjustments are made for children from 1980 onwards, but for those aged 14 years and above no adjustments have been made because for these, IQ ceased to increase in Britain (Lynn, 2009).
Where data for more than one study in a country have been reported, the mean of the two studies is given, while where there are three or more studies, median IQs are given in the last row for each nation as the best estimates of the national IQs derived from intelligence tests. IQs of multi-racial societies are calculated by weighting the IQs of the races by their proportion in the population given in Philips (1996). Descriptions of many of the studies and how the IQs are calculated are given in Lynn (2006).
Not everyone may be familiar with scatter charts and their interpretation, so in this update I am presenting the conclusions in the form of a list of the top 30 countries in 2015 PISA maths on the basis of my national IQ mediated outcomes. I also produce the data for the East Asian countries that top the PISA ‘raw results’ table, but this time in order of the raw results, but also including the place in my IQ mediated league table. Finally I give the IQ mediated UK and USA data as these are of obvious interest.
The scatter chart in my earlier article has been updated using Lynn’s latest IQ data and can be viewed alongside my ‘league table’ list for those (like me) that like scatter charts. As will be noted from what follows, the top country is Poland, closely followed by Ireland.
How I obtained my national IQ mediated list.
The first step was to rework the scatter chart using the corrected IQ data provided by Professor Lynn to produce the updated regression line. This produced a correlation between national IQ and PISA maths scores of 89 percent (compared to 88 percent previously). This shows a very strong link (remarkable even) between IQ and performance in the PISA maths test. Note that as in my previous article I have used IQ percentiles rather than standard IQ scores (mean = 100, SD = 15). The IQ percentile is the proportion of the population with that IQ score or less, so an IQ score of 100 represents the 50th percentile. IQ Percentiles can be found from IQ scores by consulting published tables. This link also contains a good explanation of IQ.
However correlation is not causation. This is very important. For example, there is a strong inverse correlation between pupil attainment in GCSE and socio-economic measures like Free School Meals (FSM). This has led virtually the entire UK educational establishment and the media to the conclusion that social deprivation causes low school attainment. This has led a whole industry from the Sutton Trust to Alan Milburn’s ‘Social Mobility Commission’ to create a class discrimination/parental skill deficit construct of an ‘Attainment Gap’, that is wholly false.
So more than correlation is needed. There has to be a credible mechanism, backed by evidence, that it is primarily high IQs that drive high attainment in school exams and in particular the PISA tests that are designed to test deep learning and sound reasoning where there is no specified knowledge content.
What does IQ measure? It is the ability to come to valid conclusions about the meaning of observations/evidence/patterns and logical propositions. In short it is general reasoning power. So is it credible that a student with well developed ‘reasoning power’ will do well on the PISA maths test? Of course it is.
If you think I am making a meal of something obvious, it has been suggested that my method lacks validity because students in East Asia that become good at maths acquire a high IQ in the process and it is this that explains the high IQ of East Asian students, rather than a high IQ makes it easier to become good at maths. Since I am a strong believer in plastic intelligence, this notion cannot be dismissed out of hand. Indeed it seems likely that the national education systems at the top of my league table do promote cognitive/IQ development. The problem is that the East Asian teaching methods, being based on rote learning from direct instruction are not effective in promoting cognitive development however much they may appeal to Daily Mail readers and the educationally illiterate politicians in charge at the DfE. This is being increasingly recognised by educationalists in those countries.
We must be clear about what a national IQ means. It does not mean that all East Asian students are cleverer than English students. The results of IQ tests produce the classic Gaussian Normal Distribution of continuous variation, as do the measurements of all individual human traits (eg weight, height) and competences.
Having established the validity of the regression line as a description of how success in the PISA maths tests varies with IQ, we can move on to what to do with it.
If you have studied GCSE maths you will know the general formula for a straight line graph:
y = mx + c
m is the gradient/slope of the line and c is the value where the line crosses the y axis. Excel works out this value for you and it is displayed on my updated scatter chart. In terms of the quantities on my scatter chart the formula becomes:
predicted score = 3.1293 x (IQ percentile) + 344.95
Note that the numbers for m and c arise entirely from the data in the chart. When I updated the chart with the revised data these numbers changed, but only slightly.
By substituting the national IQ data into the formula I obtained the predicted average PISA maths score of countries whose students have that average IQ. However the correlation is not 100 percent. Assuming that some national education systems are more effective than others the actual PISA scores are above (more effective education system) or below (less effective system) the predicted score. The underlying assumption is that had all the national education systems been equally effective then the actual PISA scores would have been the same as the predicted scores and would therefore lie on the regression line. For this to be true a large sample size is needed to allow for individual students being ill on the day/being distracted by a personal crisis, etc. The PISA system claims to provide appropriately large samples of students that are representative of the full national student population.
It is also important to recognise that a given high national PISA score could be as much down to high performance of the less cognitively able students in the sample as by the average or the most able. In other words it is not possible to conclude from the national PISA scores that high/low attainers are more/less effectively taught in one country rather than another, although deeper digging into the data does reveal such patterns, which is another reason why the PISA tests and their analysis are such a rich resource for educationalists.
To compare the effectiveness of national education systems all that is left is to subtract the predicted score from the actual score to produce a residual.
Residual = Actual Score – Predicted Score
The formula can be entered into the Excel worksheet, which then then does the calculations.
So here is my IQ mediated PISA maths national education system league table in reverse order of the residuals.
The numbers after the country are IQ, actual score, predicted score, residual
1. Poland, 92, 504, 437.9, 66.1
2. Ireland, 93, 504, 445.1, 58.9
3. Vietnam, 94, 495, 452.9, 42.1
4. UAE, 83, 427, 385.3, 41.7
5. Portugal, 95, 502, 460.4, 41.6
6. Slovenia, 96, 510, 468.6, 41.4
7. Lithuania, 92, 478, 437.9, 40.1
8. Finland, 97, 511, 476.7, 34.3
9. Estonia, 99, 520, 493.0, 27.0
10. Denmark, 98, 511, 484.8, 26.2
11. Russia, 97, 494, 476.7, 17.3
12. Qatar, 83, 402, 385.3, 16.7
13. Greece, 92, 454, 437.9, 16.1
14. Lebanon, 82, 396, 380.9, 15.1
15. Canada, 100, 516, 501.4, 14.6
16. Belgium, 99, 507, 493, 14.0
17= Romania, 91, 444, 430.7, 13.3
17= Italy, 97, 490, 476.7, 13.3
19. Germany, 99, 506, 493.0,13.0
20. Montenegro, 87*, 418, 405.3, 12.7
21. Trinidad & Tobago, 87, 417, 405.3, 11.7
22. Switzerland, 101, 521, 509.9, 11.1
23. Netherlands, 100, 512, 501.4, 10.6
24, Israel, 95, 470, 460.4, 9.6
25. Spain, 97, 486, 476.7, 9.3
26. Australia, 98, 494, 484.8, 9.2
27. France, 98, 493, 484.8, 8.2
28. Czech Republic, 98, 492, 484.8, 7.2
29. Cyprus (Greek), 91, 437, 430.7, 6.3
30. Thailand, 88, 415, 411, 3.3
* estimated IQ
The next 10 countries are in order of their actual maths score
48. Singapore, 109, 564, 572.1, -8.1
39. Hong Kong (China), 106, 548, 549.9, -1.9
45. Macau (China), 1o6, 544, 544.9, -5.9
47. Tapei (China), 106, 542, 549.9, -7.9
62. South Korea, 106, 524, 549.9, -25.9
9. Estonia, 99, 520, 493.0, 27.0
15. Canada, 100, 516, 501.4, 14.6
8. Finland, 97, 511, 476.7, 34.3
6. Slovenia, 96, 510, 468.6, 41.4
19. Germany, 99, 506, 493.0,13.0
As noted in my earlier article, there is an issue in China in relation to the average IQ levels in each area of this vast and diverse country. The Lynn data do not provide regional breakdowns. In this update the IQ figure has been revised upwards from 100 to 106, which explains why China comes out worse in my IQ mediated table.
The Global Education Reform Movement (GERM) countries.
49. UK, 100, 492, 501.4, -9.4
53. USA, 98, 470, 484.8, -14.8
Regional IQ variations probably apply in most countries and almost certainly in the UK. In England, school attainment is much lower in the former industrial northern towns that have suffered economically from de-industrialisation following globalisation. For example I have CATs data for Barrow-in-Furness that produce a mean CATs score for the town of 92. Other northern towns will be similar. In the London Borough of Hackney, the subject of my study reported in Part 4 of, ‘Learning Matters’, I give a figure of 97 for the mean CATs score.
Scotland and Wales have even higher proportions of de-industrialised, poverty blighted towns than England. It is therefore highly likely that the UK average IQ of 100 is made up of a higher figure for England balanced by lower figures for Scotland and Wales. This being the case the Scottish and Welsh governments would be very foolish not to take this into account in evaluating the PISA scores for those countries compared to England. There is no indication that the devolved governments are doing this. My advice to the Scottish and Welsh governments would be to introduce universal Y6 CATs testing in all schools. It would certainly be a great mistake to believe that the English SATs regime is a good model to be copied.
The IQ mediated league table differs massively from that published by PISA based on the raw test scores. This is to be expected. Consider a school that has Cognitive Ability Test scores (CATs) for all it pupils. Assume this school teaches maths in four ability sets. Set one contains the CATs top quartile (score of 110 or greater). Set four contains the bottom quartile (score of 90 or less). The Head of Maths wants to rate the effectiveness of the teachers of each set. Quite obviously he could not do this on the basis of the average GCSE grades obtained in each set. He/she would have to take account of the average CATs scores by using a method like that described in this article.
The current grammar school debate in England is degraded by the often deliberate failure to recognise this statistical fact.
So what do the the most effective national education systems based on my analysis have in common? PISA published a very thorough analysis of the 2012 tests. A huge range of different approaches to pedagogy were evaluated. A similar analysis of the 2015 results will doubtless soon follow. In England the Education Endowment Foundation (EEF) is funded by the government to evaluate pedagogic interventions and initiatives. The problem is that the government largely ignores its findings in favour of its free market dogma.
All of the education systems towards the top of my list are therefore worthy of study. This is not a job for me, but here is some information about the top four.
In top position is Poland. After the 2012 PISA round the Daily Telegraph published an article extolling the Polish education system quoting Education Secretary Nicky Morgan as claiming that the English education reforms were based on the Polish success. Nothing could be further from the truth as noted by PISA education guru Andreas Schleicher in the same article.
Andreas Schleicher, the OECD director of education and skills, stated: “The UK has pretty much been flat in terms of learning outcomes at least until 2012, despite a very significant increase in spending.
He said that the UK should be doing much more to check what difference education reforms make to children’s lives.
The OECD’s research also warns that the UK has some policies such as grouping pupils by ability in class, and giving families choice over schools, that could “hinder equity” – meaning they may not help to create an equal school system for all pupils.
The Polish education system has been completely reformed in recent years. The new system is described in detail here. Does it follow the English model of the creation of an artificial market in schools driven by large scale statutory testing of pupils primarily to drive league table competition between schools? It does not.
Second comes Ireland. I described the Irish education system in my earlier article. Again there is almost nothing in common with the English marketised model.
Third we have Vietnam. This may come as a surprise, but not to Andreas Schleicher, according to this BBC article.
Not just rote learning
These students are expected to leave education not just able to recite what they have learned in class, but to apply those concepts and practices in unfamiliar contexts.
In Vietnamese classrooms there is an impressive level of rigour, with teachers challenging students with demanding questions. The teachers focus on teaching a few things well and with a great sense of coherence that helps students to progress. Teachers in Vietnam are highly respected, both in society as well as in their classrooms. That may be a cultural trait, but it also reflects the role that teachers are given in the education system, which extends well beyond delivering lessons in school and embraces many dimensions of student well-being and support.
Teachers are expected to invest in their own professional development and that of their colleagues, and they work with a high degree of professional autonomy.
Respect, autonomy and high pay for teachers. No mention of testing driving marketisation. Time for a visit to Hanoi, Justine Greening. Why not take Michael Gove with you to learn where he went so wrong? The educational performance of Vietnam is especially impressive given the comparatively low level of public spending by the communist government and lack of infrastructure investment, which must be linked to the lack of an efficient tax system and the highly entrepreneurial nature of its informal and largely unregulated economy. The school system makes do with cramped buildings and at least in primary education, a two shift system in which the school week of six days is made up of half day shifts, so doubling the number of pupils that can be enrolled in each school. Education, however, has a very high priority.
Fourth is UAE. This really is a surprise. Once again we have a country that has recently reformed its entire education system and certainly not on the UK or East Asian model. UAE school students have an average IQ of 83 (13th percentile). Compare this with Singapore (73rd percentile). Yet the UAE students achieved a PISA maths score of 427. What PISA score was being achieved by the very low proportion of Singaporean students with an IQ of 83? What would be their chances of passing the high very high stakes national exams in the Singapore and Chinese systems? All students in these East Asian countries appear to have desperately pressured school lives. This is not good for cognitive development.
This is from a publication of the Embassy of UAE in the USA
Education reform focuses on better preparation, greater accountability, higher standards and improved professionalism. In addition, rote instruction is being replaced with more interactive forms of learning [my bold], and English language education is being integrated into other subjects, such as math and science. The Abu Dhabi Education Council (ADEC), the Dubai Education Council (DEC) and the UAE Ministry of Education (MOE) are each tasked with education reform, while preserving local traditions, principles and the cultural identity of the UAE.
Education at primary and secondary levels is universal and compulsory up to the ninth grade. This takes place in a four-tier process over 14 years:
- 4 to 5 year-olds attend kindergarten
- 6 to 11 year-olds attend primary schools
- the preparatory stage caters for children aged between 12 and 14 and
- 15 to 17 year-olds attend secondary schools.
A note of caution is needed here. The fourth place of UAE is very much down to the very low IQ score of 83. This is from a single study.
So what can we say about the UK (49th) and USA (53rd) systems?
There is clearly very little to be positive about that is for sure. Even more depressing is that the frantic pace of reform is to be stepped up with more testing, more Academies and Free Schools, more faith schools with their own enhanced sectarian admissions rules and now the imposition of selective grammar schools. It would be hard to come up with proposals to make the national education system worse.
The most important message to the DfE is the key role of cognitive ability in driving higher attainment. This needs more of the well-proven developmental pedagogy that the ideology of marketisation is replacing with knowledge-focussed rote learning and behaviourism, enforced by ever more draconian and abusive systems of harsh discipline.
The potential for raising standards through exploiting the potential of ‘plastic intelligence’ is explained here, and the dire consequences of further attempts to ‘close the gap’ are set out here.
This article describes how cognitive developed can be enhance through fundamental changes to school culture.
And finally if we are looking to the long term, this article explains how Chinese and other East Asian countries came to have such high national IQs.
I am very happy to discuss any issues related to the validity of the arguments/data in this article either through ‘Comments’ or privately at:
Roger Titcombe@yahoo.co.uk
Roger Titcombe's Learning Matters 
You indeed use very old data. Check this paper: https://openpsych.net/forum/attachment.php?aid=552
Published in Open Differential Psychology, 18th November 2014.
Fluid g in Scandinavia and Finland: Comparing results from
PISA Creative Problem Solving and the WAIS IV matrices
subtest by
Edward Dutton and Emil O. W. Kirkegaard
Finns have IQ 100 in Matrices, UK scaled to 100, based on recent test standardization data.
The paper derives IQ also from PISA. Differs from your conclusions.
Hope this helps,
LikeLike
Thank you for sharing this very interesting paper. I take the point that I could have got the mean Finnish IQ wrong. If it is indeed 100 rather than 97 (on the UK scale) then this will push Finland further down my IQ corrected PISA maths league table. However Finland will still be doing better than the UK because the raw PISA scores will then be directly comparable (UK – 492, Finland 511).
As I make clear in my article, the validity of my corrected national league table depends on the validity of the national IQ data. The Dutton and Kirkegaard paper shows the complexity of establishing such data and the variety of explanations that they generate for variations in national IQ. In comparing the higher Finnish mean IQ with the rest of Scandinavia the authors accept that the superiority of the Finnish education system is a tenable explanation in terms of a positive Flynn effect (compared to the negative Flynn effect for the marketised English education system recorded by James Flynn himself and Michael Shayer).
There is nothing here to challenge my assertion that national IQs have to be taken into account in order to draw conclusions about the effectiveness of national education systems from raw PISA international test results.
The most important conclusion that flows from my analysis is that the very high raw PISA results of the East Asian countries are a consequence of the very high mean national IQs of those countries, rather than the effectiveness of their education systems, which actually come out very badly on the basis of my IQ corrected residuals. This is of the utmost importance here in England where there is government pressure to force our already poorly performing schools to emulate Chinese and Singaporean teaching methods, which would clearly make things much worse.
Fortunately for my argument, not only are the East Asian IQ data robust, but even more importantly the same high IQs are found in UK data for British children of East Asian ethnicity attending UK schools.This can only be explained by a genetic contribution to the high East Asian IQs.
I speculate about a possible explanation for this here.
LikeLike
I cannot concentrate on your article right now, it is interesting but I will read it later. It may solve the whole problem, but I cannot comment it now.
I think it is shown clearly enough that IQ has a genetic component and also that the main races differ in this aspect partially because of genetic reasons. About the Finnish PISA results, there is a study showing differences by ethnic groups in Finnish schools. Apparently the Finnish teaching method is good mainly for Finnish girls. Finnish-swedes get a bit lower points, somalies get much lower point also in Finland, boys get lower than girls. All one can say of the Finnish example is that each teaching method fits some specific group best. For instance, in Finland teachers of mathematical departments in universities have not been so convinced of the greatness of math teaching in Finnish schools. The results may look good in PISA, but I have been told a few times that universities today have to teach issues that used to be learned in secondary school. So, maybe the Asian way of teaching has some strengths also, at least for boys and in math.
There is genetics involved, that is true. I looked at this 2D:4D finger index, and though Finnish index is not as low as Manning announced, it is lower than in the geographical position should be. It probably reflects Asian genes, Chinese have similar results. Possibly there is some connection to spatial IQ, though some see it, some do not. Teaching should be adopted to the population.
LikeLike
Hello J2, You write, “I think it is shown clearly enough that IQ has a genetic component and also that the main races differ in this aspect partially because of genetic reasons.”
Yes, and unless this is accepted and taken account of, it is not possible to validly compare national education systems from the raw PISA test scores, which is implicit in my article.
However, if you read other articles here on my website it will be clear that I believe that the main focus of national education systems should not be tailoring teaching approaches to the mean IQs of particular ethnic/social class groups, but rather in recognising the plasticity of intelligence and the potential for raising it for individuals and groups through the right kind of teaching. See
LikeLike
You may be right, but what do you think is the goal of teaching on school level? Keep the country economy going on, or something more valuable/traditional? Maybe the goal today is simply to fill meters like PISA because superiors have adopted management methods where they are following meters and the result is what is measurable. Like in a university the goal may be the number of masters, doctors, publications, references, what ever sense there is in those results. I have not been in any school recently, maybe it is the same there. That is, why the UK PISA results should be any better?
LikeLike
Surely the point of PISA is to compare the approaches to teaching and learning in different countries so that good practice can be disseminated so learning can become more effective for students. For me this means developing cognitive and other abilities, not just passing exams. National education systems can be very different. See this article about the Finnish system. For this to happen it is important that PISA results are validly interpreted, hence this article,
LikeLike
Are you aware of the work of Professor Feuerstein – Adey and Shayer who you cite quite prominently are amongst the small number of people that explicitly acknowledged him.
Some important info is here:
https://www.linkedin.com/pulse/recent-audit-cognitive-programs-university-auckland-james-colvin
LikeLike
You are right James. There is indeed some important work going on that I was unaware of. Please keep in touch @rogertitcombe
Many thanks
LikeLike
I will read your article, will take some time. About PISA, yes, generally speaking what you wrote, that surely is the goal in PISA. What I meant were the choices that appear once one goes to a more concrete level.
1) If the goal is that a pupil can continue his/her studies in a university or some other educational institute and some fields are considered more useful (say engineering, business, medicine), some less, then school may give good starting point for such desired studies, which may for instance mean that math should not be taught by text form questions what PISA likes, but more emphasis should be on solving equations, traditional math, because in continued studies one never gets to the practical problems. You cannot rewrite the whole math/science in word form.
2) If the goal is that after school the pupil has congnitive abilities that in some way fit to the life he is living, then the math problems should be practical and interesting (to those who do not like equations), and in general the studies should not be preparing to other studies that are still (and will always be) traditional.
3) and if the goal is that the pupil has a wide range of possibilities to continue the studies in whatever field, or for instance start a firm, the structure of studies should be different. I have no idea what that would be.
So one could find the best practices by looking at the selected meters?
What are the best practices, and for what goal? I am afraid that what is measured improves, meaning that measured things become the only practical goal. Other things will be mentioned only in talks.
From your artice I understood that you did not like the idea of UK using East Asian teaching methods. Possibly the East Asian teaching method is quite good for something, maybe for continued studies abroad.
How I see the Finnish system? When comparing Finnish students in the seventies or eighties and now, I mean some ten years ago, I think they are much more grown up, more self-assured, not worse at all than in the old times, much better in my opinion, but then there are teachers of math who think that the freshmen do not any more manage to solve equations. What is good in the Finnish system is that there are no elite schools or elite universities, so it is more equal. That has a negative effect on the number of Nobel prizes, but that is not so relevant.
I think education goals are some kind of politics, or choices, some people have selected to goals, and it is not as clear as in your fine answer.
LikeLike
Interesting that a country like Portugal that has a Nat iq according to Lynn’s data as being around 95 can somehow outperform Spain and other supposedly higher iq countries. Why do you think that is? Maybe the Flynn effect has caught up? We know that Portugal had very high levels of illiteracy among the 50+ old pop, maybe it depressed the avg iq in older samples. Certainly the younger generation seems to consistently have better scores on the TIMSS and PISA than Spain. The same can be said for Ireland.
LikeLike