Better by leagues?

Guest blog by
Dr Kevin Denny
School of Economics, UCD
Geary Institute, UCD

Reading Saturday’s Irish Times I came across a feature by Kate Holmquist with the sub-title ‘Many parents are questioning whether league tables can give a full picture of the educational quality provided by different kinds of school.’ It’s an interesting and important topic and I assumed that this was a piece on other countries experience of league table. I was surprised that it was, in fact, about Ireland. The small problem here is that we don’t have league tables in Ireland. Their publication is, as Kate mentioned, illegal under the Education Act. What was being referred here were the tables of ‘feeder schools’ published by the Irish Times and whose publication is eagerly awaited by parents.

While I don’t wish to pick on this article in particular, it was like most of the ‘debate’ on the topic: one-sided and rather superficial. Discussions of this topic are monopolized by those opposed to league tables; they simply rehearse the same old arguments, but without considering alternatives seriously, and they ignore the experience of other countries and indeed the substantial research literature on the subject – some of which might actually support their position.

The basic argument is the following: school is multi-dimensional, it’s not all about points. Such tables don’t reflect that. Ergo, nothing is better than something. Since deciding what school to send their children to is one of the biggest decisions many parents make it is fundamental that they are in a position to make an informed decision. Hence we need a balanced and informed discussion of this topic. During the Celtic Tiger there was a movement away from public schools to fee-paying schools, a trend which seems to have been reversed recently. Parents believed they are getting more by moving their kids to the private sector and for some parents the outlay on fees is considerable. But is it justified? Are fee-paying schools all that good and are the state schools all that bad? Probably not, but parents cannot know for certain because the information that might help them decide is censored. Ireland, incidentally, is out of line with other English speaking countries in publishing no information on school performance.

Making an informed decision is precisely what the educational system seems determined to prevent. Those who argue against providing exam and other data on schools are implicitly saying that they know the limitations of this information but that the public does not. In other words, parents and students are not intelligent enough to understand the information and need to be protected from their own ignorance. This argument is both baseless and condescending.

If the government were to be consistent and apply this argument more generally, then the public would be kept in the dark about an awful lot more. For example, the government and its agencies publish large amounts of data on the economy, such as inflation, growth, earnings etc. Such information is routinely misinterpreted by journalists, politicians and sundry other commentators. It’s an occupational hazard if you are an economist and doubtless I commit similar errors in other areas. However no reasonable person would argue that the public should be kept in the dark on these matters.

Look at the packaging of food in your kitchen: much of it contains details of the nutritional content; however most of us have only the vaguest notion of what it all means. Should we censor it? No, because there are people who do understand it, and in a democracy people make up their own minds.  A good analogy is how we measure the economic performance of a country. The best known indicator is GNP per capita and everyone knows that this is limited. It takes no account of non-market activity or the impact on the environment. However we do not censor this data, we supplement it with additional indicators. For example, economists have been developing ‘Green accounting’ which takes environmental factors into account.

Curiously, successive governments are in favour of ‘crude league tables’ when it suits them. For many years the OECD has been collecting comparative data on students’ performance, for example the PISA data. When this data shows Irish students in a favourable light, the educational establishment is only too happy to point this out as a ringing endorsement of our educational system. But these international league tables are every bit as crude as comparing Leaving Certificate results across schools. They take no account of differences in funding, curriculum, teaching methods or anything else across countries.

The principal argument against providing information on exam results by school is that this is a crude indicator of what schools do. It is argued that education is multi-dimensional and schools do more than simply send students out to the world with their Leaving Certificates. In addition, the exam results take no account of the inputs: schools vary both in the resources available to them and the composition of the student body. In economic terms, exam results are a measure of ‘output’ rather than ‘value added’ which is what we want to know. While these points are have some validity they provide no justification for the current impasse. The intelligent response to crude league tables is to have sophisticated league tables. That is, since there is more to schools than just exams this is an argument for providing more information, not less.

There is no reason why a comprehensive set of indicators cannot be provided on key aspects of what schools do.  Priority should be given to those factors that are most important to parents and students, and for them, whether we like it or not, academic performance is a priority. We know this because the 2004 survey by the Education Research Centre showed very clearly that a significant majority wanted more information on a whole host of academic variables including examination results. In addition, information on the number of migrants and special needs students in a school could be included since this can affect results.

The question of measuring ‘value added’ has been dealt with in England which now provides value added tables for secondary schools based on the improvement in students’ performance over time. An easy way of doing this in Ireland is to use the two state exams. Comparing the difference between the performance at Leaving and Junior Certificate, at the level of the school, gives a simple measure of value added. It is not perfect – no measure is – but it has the advantage of being easily calculated and understood. To avoid the most invidious comparisons, a sensible strategy would be to take an average over, say, a three year interval to smooth out excessive variation and schools could be grouped into say twenty bands, the top 5%, the next 5% and so on.

In our Information Society, knowledge is a key currency and access to it is highly unequal. Many individuals, those of us with good social and professional networks, probably have a good idea of where the good schools are. But on grounds of equity, everyone should be well informed. There is a whole body of science devoted to measuring, presenting and interpreting information scientifically. It is called Statistics, and it is time the educational establishment put it to use to serve the public interest rather than making excuses so as to defend the status quo.

Explore posts in the same categories: education, society


You can comment below, or link to this permanent URL from your own site.

12 Comments on “Better by leagues?”

  1. Vincent Says:

    If as you say it is part of the Act and the information exists then it is wide open to challenge under about twenty different sections of the European Human Rights. Girls versus boys being the most blunt, to the argument that the fee-paying schools by their nature share such info’ between themselves and potential clients it is blatant discrimination.
    And anyway all of this can be pulled via the Committee in the Dail over the head of the Act as the entire thing is a public account paid by Vote.

  2. kevin denny Says:

    Rather than getting the law involved, we need to approach this positively and not get into a “parents vs. teachers/schools” situation which polarizes opinion & puts people on the defensive. We are all on the same side I hope.
    So the challenge is to provide parents and students with the information that they need -and clearly want- in a way that allows to make them informed decisions while avoiding some of the problems that have been well aired. It certainly wouldn’t help if some school got labeled “worst school in Ireland”.
    These issues are not unique to Ireland, other countries have grappled with them and come up with better solutions.

    • Vincent Says:

      And why the hell not, I ask you. If one then can see where a focused charge of cash can do some good.
      All this pussy-footing about makes me want to puke. How would you like it if ten years down the track you find you have sent your sprig to the educational equivalent of a Texan public defender when you could have sent it down the road to the public school.
      Further, given a few hours, with a bit of Mental Reservation, I could find for the private sector exactly who what when and where. And I could probably get some to contrast with Ampleforth, just for the fun.
      So, it is not that the info is not available for some, but that there is a discrimination. And with the best will in the world, Kevin. The Courts or the Dail Public Accounts are the only ones that can split the knot.

  3. Iainmacl Says:

    Ah now…firstly there are other ‘other countries’ than England, you know. Just because it presents a dreamland for those who believe in meaningless managerialism and the deprofessionalisation of public services, doesnt mean it appeals to all of us! 😉

    Seriously though, there are other countries which have abandoned league tables and related approaches. I guess there are two basic choices when faced with a system that clearly doesnt work (league tables as a means of educational improvement): (1) recognise it’s rubbish, scrap it and address the real issues head on; (2) try to gather even more data at more cost and effort to chase the ‘holy grail’ of the final set of measures that really work.

    Also, objection is not about pussy-footing, Vincent, there are real issues here and not just in the intrinsic difficulty of measurement, but more basic points about a lack of leadership – it doesnt need statistical measures to identify if a school is doing badly or well…Secondly, there’s the underlying philsophical position that surely what we need is a system that ensures that all children get the best possible education and not just that the well off have a means of identifying what the ‘best schools’ are to send their children too. Public lynching in the press via league tables doesn’t improve the system for the kids already in those labels them, and makes the position worse. What’s needed is real leadership and action to ensure that all schools are properly resourced and managed with an ethos that celebrates learning and achievement. With league tables, we’re likely to head further to the situation that Robert Reich memorably described as ‘secession of the successful’….

    • Vincent Says:

      I sorry but it is a standard test across the State, but it sure as hell is not equal in the inputs, and the CAO has little intrinsic difficulty. And if you are measuring the kids that do the leaving cert with this test then why on earth not publish the results. The only important bit of info that should be protected is the Names of the kids.
      Let us switch the another group that decided in the dim and distant that Stats were prejudicial, hospitals. Now, if given the choice would you have an operation at some medical facility with a high failure rate -read deaths- or one where survival is near 100%.
      With the schools what matters is the Leaving, with the kids what matters is the leaving and with the Universities what matters is the leaving. Now if the school cannot get the potential out of the kids, it should not exist.
      And why exactly are we protecting Teachers at the expense of the kids.
      But again this information is available for a section of the schools, it is part of their selling point. And granted they might not tell you how many non-pass they have, they will have no problem telling you how many where and at what they had over the past ten or so years.
      So, what do we call it when a section of a population has advantage over the majority.

    • kevin denny Says:

      Information on school performance is available in England, Scotland, Wales (I think), US, Netherlands, Norway, some provinces in Australia & Canada. For language reasons I can’t check other countries. I would say their abandonment is the exception rather than the rule. In the North they were decommissioned by Martin McGuinness.
      So we are they exception.

      There are difficulties about measuring anything complicated but that’s no excuse for doing nothing: there are ways of dealing with it, they are not expensive at all and I suggested some. Measuring economic performance or inequality or environmental quality isn’t easy but we don’t make it illegal. Why? Because its important and people have a right to know.
      The point is that while we know some schools are better than others we don’t have accurate measures, people will go on hunches and rumours and prejudices (like “if its expensive it must be good”). The idea that keeping people in the dark, when they plainly do not wish to be, can somehow lead to better decisions is self-evidently nonsense in my opinion. In

  4. Hugh Says:

    I’m not an academic and no longer have any direct involvement in our education system, so I confess I have no in depth knowledge of this subject. Nevertheless, I suggest that debating about league tables puts the cart before the horse. Why measure one metric of something that isn’t working as well as it should? It seems to me that our system is not currently producing well-rounded, engaged and thoughtful members of society. Yes, of course there are exceptions, but I’m discussing this in general terms. The system is geared to producing academic achievers. That’s what the state examinations measure, and that’s what parents appear to be focused on. Universitydiary has been reminded of this recently – – and many students seem to be unable to focus on anything else. As measures of academic achievement, league tables seem to hit the spot. But aren’t we missing the point? To use one of your analogies, what good is a food label that tells me the product I’m about to eat has 0% fat, but fails to tell me about all the salt, MSG, preservatives and antibiotics that are in there as well? Not only is it no good, its potentially very misleading.

  5. Niall Says:

    I’m with Kevin on this one. Information on academic performance should be made available to the general public – not making it available is discriminatory as then, only those ‘in the know’ have the information.

    Iain and Hugh say the system doesn’t work. How do they know this in the absence of public information?. How do we identify failing schools without measurement? I am in absolute agreement that all children should get the best possible education. However, I’m not such a snob to think that the well-off are alone in wanting good schools for their children. You only have to look at the growth of the Educate Together and Gaeilscoileanna in Ireland to see that is simply not true. Good schools can of course be defined on more than exam results e.g. ethos, sports, extra-curricular activities. Information on these should be publicly available too – as should info on special needs. This would certainly be of interest to parents of children with special needs – well-off or otherwise.

    • Hugh Says:

      I said that it isn’t working as well as it should. My particular hobby-horse is that teaching has for years been focussed on preparation for state examinations to such an extent that students are not learning how practically to apply what they’re learning in the real world. They’re not being required to think for themselves, their focus seems to me to be unacceptably narrow, the kind of league table we’re discussing reinforces narrow focus, and we’re getting to the stage where students will lose the sense of wonder about the world that’s essential for creativity and progress. Really, if I were making enquiries about a school, its ranking on this kind of league table would carry very little weight. I’m not saying they’re of no use – I’m saying there are important issues to be addressed first.

      • kevin denny Says:

        Hugh , while I agree with much of what you say I’m very puzzled by the question “Why measure one metric of something that isn’t working as well as it should? ” So should we not have metrics of the health service, the economy and well, pretty much everything?

        There are 2 issues, what should education be doing and what information should be available. But we have to work on these together.We can’t wait to get the first sorted out before addressing the second (or vice versa) because we will be waiting for ever. And also the process of providing greater transparency will help us move forward. If we start with indicators of some things (like what we measure already) then maybe we can move to the trickier stuff that you allude to. But I don’t see how keeping people in the dark can ever be good especially when the vast majority of people are quite clear that they want to know.
        The other point is that you may think that people are too focused on points, say. I might agree with you. But in a democracy people have a right to their opinions so even if I don’t like it, thats tough. Personally, I think that people are too interested in celebrity television programs, Tiger Woods’ love life & GAA. But I would never censor that information.

  6. […] Some of my thoughts on school league tables can be found here. […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: