In this game is the REF to blame?
Anyone working in and around higher education in the United Kingdom will have been obsessing about the ‘Research Excellence Framework’ (REF) over the past week. According to the REF website, it is ‘the new system for assessing the quality of research in UK higher education institutions’. A total of 154 institutions made 1,911 submissions to this exercise, and last week they found out how they had fared. The results will influence a number of things, including league table positions of universities and public funding. They will also have reinforced a trend to focus research attention and funding on a smaller number of institutions.
REF is the successor to the Research Assessment Exercise, which in turn had been around since the 1980s. The first one of these I had to deal with was conducted in 1992, when I was Dean of the Law School of the University of Hull. While I believe I was rather successful in managing the RAE, in that my department improved hugely between 1992 and the next exercise in 1998, I now believe that most of the decisions we took were good for the RAE and bad for research. In fact, that could be the overall summary for the whole process across the country from the beginnings right up to last week’s REF.
And here are three reasons.
- The RAE and REF have, despite claims to the contrary, punished interdisciplinarity, because the units of assessment overwhelmingly focus on outputs within rather than between disciplines. The future of research is interdisciplinary – but academics worried about REF will be wary of focusing too much on such work.
- Despite the way in which it aims to reward international recognition, a key impact of the RAE/REF framework is to promote mediocrity. For funding and related reasons, many institutions will try to drive as many academics as possible into published research, spending major resources on pushing average researchers to perform – resources that should really be devoted to supporting those who have the most promise. Of course some excellent researchers have been able to thrive, but in many institutions the RAE/REF process has hindered rather than supported real excellence. On top of that it has diverted some staff from doing what they do really well into doing things they don’t much like. One of the casualties of that, incidentally, is collegiality.
- The RAE/REF has produced a stunning bureaucratisation of research. A key difference between research management in my last university in Ireland (where there is no such exercise) and in my current one is the extraordinary amount of time staff have to put into the tactical, operational and administrative maintenance of the REF industry. Also, I shudder to think how much time and resources institutions will have spent last week managing the news of the results. Industrial-scale bureaucracy of course also produces huge costs.
Other equally good reasons for doubting the value of REF have been given by Professor Derek Sayer of Lancaster University, writing in the Guardian.
I am not against competition in research, nor do I believe that research performance should not be monitored. But the RAE/REF process is about ranking universities rather than promoting research. I have no reason to think that anyone who matters is listening, but it is time to think again about this process.
Explore posts in the same categories: higher educationTags: Derek Sayer, RAE, REF, research, research assessment exercise, Research Excellence Framework
You can comment below, or link to this permanent URL from your own site.
University Blog 
December 23, 2014 at 8:50 am
“I have no reason to think that anyone who matters is listening….”
This is not the least offensive statement you have ever written.
December 23, 2014 at 10:53 am
“The RAE/REF process is about ranking universities rather than promoting research”
Indeed one could say it is about ranking universities rather than ranking research, such latter purpose is not achieved since it has been argued that for REF reviewers it is impossible to go through the material to read, not to mention matters of their own expertise. The system won’t change until there are winners and losers and the benefits of the few will prevail over what is best for the whole sector. That’s the logic of a *competitive* university system.
December 23, 2014 at 12:13 pm
I’m not getting into this.
Happy Christmas to you and all of yours Ferdinand. And best of luck for the ’15. Jeepers when did that happen, it’s only yesterday you went over to Scotland.
December 23, 2014 at 4:35 pm
Thank you, Vincent. And yes, time flies.
December 23, 2014 at 1:02 pm
Excellent post, Ferdinand. I saw Professor Sayer’s piece, and many UK scientists whose work I hugely respect have been saying something similar for some time now.
Re “I now believe that most of the decisions we took were good for the RAE and bad for research”, I would like this courageous statement from a very senior figure in academia to be heard as widely as possible. Would you consider an article in the THE of the FT? It could make a difference…
merry christmas, Cormac
December 23, 2014 at 4:36 pm
Many thanks, Cormac – and yes, I shall certainly think about pressing this message to a wider audience. On the whole VCs are solidly pro-REF…
December 23, 2014 at 1:04 pm
Only those universities which do not do well mumble and grumble. Not every university is fit to do research anyway. They are mostly teaching only universities. I like REF and our two big universities in London-Imperial and UCL have done well. Well done guys!!
More of a problem is the dramatic slump in oil price; the SNP is getting quiet about another referendum go, and Sturgeon uttered the dreaded “U” word yesterday-“UK”, she knows what this slide in oil price means! NE Scotland is fearing heavy oil industry -related job losses on a massive scale. I would worry about it and not REF.
December 23, 2014 at 4:53 pm
I have to say, ‘Eddie’, I am really disappointed with you. You neglected to say ‘In this rarely read blog’. At Christmas of all times it is important to keep up tradition.
December 23, 2014 at 5:03 pm
I am not disappointed at all with your tradition of : ” I know best and here is my unsolicited advice”. which you are dishing out.
December 23, 2014 at 2:47 pm
It is simply not true that “only those universities which do not do well mumble and grumble”, Eddie. Colleagues, co-authors and friends of mine at both Oxford and Cambridge University complain bitterly about the time they invest in documentation for the REF…they have no problem with elitism, but major problems with this particular process
December 23, 2014 at 5:07 pm
I have equally good number of friends in the universities you mention, who like the REF process. There are two sides to REF, and I will leave it at that.
December 23, 2014 at 3:36 pm
There is a very good summary of why the REF process does not deliver an accurate picture of research in UK universities at
http://theconversation.com/game-playing-of-the-ref-makes-it-an-incomplete-census-35707
December 24, 2014 at 9:29 am
You mentioned you worked at a University in Ireland, I’m interesting how they allocate funding?
I’ve read a lot about the faults with the ref (and i was involved in the submission process at my own University, so know many first hand), but haven’t see much about the possible alternatives.
It seems to be that if you have one Chemistry department doing fairly good research, and another, through innovation, long hours, developing staff, careful spending etc etc that produces fantastic research, then it makes sense to direct more funds to the latter, not only to get more ‘bang for buck’ but to encourage departments to do so. The question is how to do this in an efficient manner?
December 24, 2014 at 10:59 am
This seems to be not so much a grumble about the results, but more a belated recognition of how your own university got it wrong – expending its resources on admin instead of backing its best researchers means that those researchers will be handicapped financially and in many other ways for the next funding period – if they stay with you.
But perhaps you also need to recognise that it’s a reputation for being at the leading edge that attracts students and sponsors. Regurgitating someone else’s work in a classroom will never be as attractive. And that means pushing everyone to perform better in the public arena of research. You don’t need an admin framework for that – just vision and resolute leadership.
January 3, 2015 at 1:09 pm
It is not the REF to blame, it is certainly the wrong decisions made at the senior level, the lack of support to research, the culture and mindsets at some institutes. The REF results should be a wake up call, and the message is clear ‘we are doing a bad job in terms of research’, we need actions, radical changes, clear plan with well defined goals and milestones. No gain from this REF-blame game
January 8, 2015 at 9:17 pm
Belatedly, my take on REF in 103 words, bit droll 🙂
http://www.timeshighereducation.co.uk/comment/letters/seats-of-learning/2017817.article