In the United Kingdom at least there now appears to be a belief that assuring quality means measuring things. This, as we have noted previously in this blog, lies at the heart of the Research Excellence Framework (REF – previously the Research Assessment Exercise), and it appears increasingly likely it will also be at the heart of the proposed Teaching Excellence Framework (TEF). In fact these exercises tend to consider an intriguing jumble of inputs and outputs and put a relative value on them. The result is seen as a kind of gold standard. The REF in particular is viewed not just as a table of research excellence, but also as some sort of indicator of wider institutional quality. Few seem to think, as perhaps they should, that such massive exercises will often prompt worthy mediocrity much more than intellectual creativity. And nobody much seems to want to ask why virtually no other country thinks this sort of thing is a good idea.
But maybe our fatalism that this is inevitably our destiny might be shaken a little if we thought more about the cost of it all. The journal Times Higher Education has recently referred to studies suggesting that the cost of the most recent REF may have been anything from £214 million to £1 billion. To put that into some sort of perspective, even the smaller of these figures is nearly as much as the entire annual funding of all of Ireland’s universities (including tuition fees paid by the state). For this kind of cost to be worthwhile it would have to guarantee an enormous explosion of research excellence producing massive educational and financial benefits to the institutions and to society. There is really no evidence to suggest this is the case. The history of the RAE and REF does show they prompted a much greater volume of publication, but there is no evidence at all that this generated a greater amount of innovative discovery or scholarly insight. In passing it can be said with some assurance that research funding does have such an impact, and competing for it produces benefits – but no such claims can be proven to be true for REF. And now we are apparently about to load another huge cost on to the system in TEF, almost certainly with similarly uncertain benefits.
We do need to secure high quality teaching and research. But we also need to display much more sophistication as to how this can be assured.