Taking charge of your own university rankings
Whenever I raise the topic of university rankings, I always get readers who, either in comments made here or in emails sent offline, will suggest that I really shouldn’t be paying so much attention to them or encouraging their authors. I know very well that many academics are very sceptical about league tables and don’t believe that they reflect any sort of reality; or they suspect that rankings prompt inappropriate behaviour by university managers, or in some other way undermine academic integrity.
In reality, however, league tables are part of the landscape, and this is so in part because those who want to enter into any kind of relationship with universities – whether as students or as faculty or as business partners or as donors – take them seriously and want to have them as a guide. We may wish that it were otherwise, but it isn’t. This being so, we need to engage with them, and in that way help to ensure that they are reasonable and accurate and transparent. So for example, the transformation over the past year or two of the Times Higher Education world rankings owes a lot to academic interaction with the journal and with the company they selected to manage the project.
The best known world rankings – those run by Times Higher Education and by Shanghai Jiao Tong University – have one important thing in common: the global top 10 universities are exclusively American and British. This is tolerated by Asian institutions that believe they are rising up the tables and are biding their time, but it disturbs the European Union and its member states. In both rankings the top non-British EU university only comes in at number 39 (French in each table, but not the same university).
Because of this the EU has set out to design its own rankings, to be known as U-Multirank. The thinking behind this is that the established league tables are too much focused on research outputs, and in particular on science research; they neglect teaching and don’t encourage diversity of mission, and they drive universities into strategies that they don’t have the means to deliver. So the new rankings are to be weighted differently, so that the resulting table would be more balanced; and moreover they are to allow users to design and weight their own criteria, so that students (say) can create their own league table that more accurately reflects the strengths they are looking for in considering universities.
Can this work? In my view, no – probably not. Rankings are not really meant to provide a method of institutional profiling, but rather are designed to set out a kind of reputational gold standard. They are not the answer to the question ‘what kind of institution is this?’ – rather, they answer the question ‘what does the world think of this institution?’ This may not be a scientific answer, or else all rankings would give us the same results, but it is an attempt at standardising external evaluation. Also, too many people will think of U-Multirank as an attempt to support the somewhat lesser known European universities and design the rules to suit them.
Still, if you’re interested, the U-Multirank project is coming to the end of a feasibility evaluation and, if this supports the idea (as it will), it will be rolled out some time over the next year or two. It will be interesting to see whether it attracts support. I suspect that it will not displace the pre-eminence of Times Higher.
Explore posts in the same categories: higher education, universityTags: European Union, Shanghai Jiao Tong, Times Higher Education, U-Multirank, university league tables, world rankings
You can comment below, or link to this permanent URL from your own site.
University Blog 
April 15, 2011 at 1:18 am
Personally I think there is a danger of being distracted by rankings.
Especially here in Ireland….
Now that we have to castrate our potential.
But let’s say we find serious amounts of oil in a years time?
We have Norwegian levels that can fund whatever…
Where is Norway on the league table, with its wealth?
April 15, 2011 at 2:06 am
In fairness, might not the difficulty be that those inside the Uni’s and those outside are reading different things from the rankings. Whichever of them. No one really cares that Loughborough’s output in the dept of Widgitiwambs equals Oxford. This is like beer. Most people who visit will drink Guinness in Ulaanbaatar. It will not matter that they have a perfectly good Chinggis Beer. Nor will it matter where the visitor calls home or that the visitor never sipped a Guinness before.
To put it another way. Oxford could decide collectively to go on the razz for five years, taking in no students and publishing nothing, without moving more that a few places. And by the same token, could decide to go back to using Latin for all contact with students and Greek between fellows. And have such an action read plausibly.
April 15, 2011 at 6:09 am
[…] “Whenever I raise the topic of university rankings, I always get readers who, either in comments made here or in emails sent offline, will suggest that I really shouldn’t be paying so much attention to them or encouraging their authors …” (more) […]
April 15, 2011 at 9:05 am
Nothing annoys me more than university ‘rankings’ because we’ve basically taken the Channel 4 lists of top comedians, children’s toys, etc and applied that form of cheap, lowest-common-denominator entertainment to the university sector. To borrow a quote, these rankings are based on known-knowns (how many students are enrolled, what is the average publication output per academic member of staff, etc), known-unknowns (what is the student-to-staff ratio based on university definitions of ‘academic’ staff, etc), and unknown-unknowns (what do employers think of the university graduates and how do students rate their education). We then apply ‘weightings’, or in mathematical terms ‘guesses’, to how important they are and fiddle the numbers until Oxford, Cambridge, MIT, Caltech, Harvard, etc come out near the top. If different measurement systems give you the same top twenty universities then we can conclude that those universities are good. If everything else is a muddle then the measurement system doesn’t work beyond the top twenty and they should be dismissed as ‘outliers’ on the curve of a system with limited applicability.
I disagree with the premise that these rankings are used by anyone other than government agencies. Companies don’t invest in universities on the basis of rankings. They invest because governments give them complicated tax benefits if they do. Multinationals invest in ‘research institutes’ in Ireland because their research is in the right area, the companies have local manufacturing plants and will therefore invest in local universities, and they want a nice steady stream of local graduates so they won’t have to spend time, money and effort recruiting from abroad. If ‘rankings’ really were important to companies then they would only employ Trinity graduates, and since observation doesn’t match that hypothesis, the conclusion is rankings aren’t important for employers in Ireland.
The public and students certainly don’t use the rankings – in the UK and Ireland rankings have very little to do with college choices. Instead students look at courses, decide if a university offers a course they’re interested in, look at the entry requirements and whether they’ll meet them, listen to the experiences of friends and relatives who went to those institutions, and then assess how many trains, trams and buses they’ll need to take to get there for 10 am on a Monday morning. Geographical location determine student choices to a greater extent than rankings.
So we’re left with government agencies who use rankings as a way of judging the effectiveness of their funding and PR staff who like to splash ‘WE’RE IN THE TOP 100’ on their websites. What we should be asking is, if rankings ARE being used by governments, then why are soverign states using statistics compiled on behalf of a newspaper by a private company rather than internal metrics as a means of evaluating that performance?
April 15, 2011 at 7:59 pm
well said!
April 15, 2011 at 9:33 pm
Terrific post!
April 15, 2011 at 11:34 pm
jfryar, you may want to believe this, but it isn’t the case. Students do use the rankings – we get evidence of this all the time – and industry certainly does. Ironically, I don’t actually think that government does, much.
April 16, 2011 at 7:53 am
Well, I was involved in a DCU survey conducted by the Faculty of Science and Health about three years ago as part of their ‘marketing group’. It involved students at Open Days and also followed up with the first years. The conclusions were simple – geographic location and the good experiences of friends/relative were the primary reasons people chose DCU. That data should still be available from the Faculty.
Yes, some students may use rankings, but my point is that student choice of university is a complicated beasty and rankings are not, at least according to any survey I’ve seen, the key determining factor. It may be a factor for international students, but I’m pretty sure the concensus will be a university education in a lower ranked university is better than no education at all. In which case rankings are not as important as the entry level requirements.
If industry uses rankings then why do some invest in DCU and/or UCD, rather than Trinity? Unless, again, their reasons are complicated and rankings are only a minor consideration in that scheme.
April 15, 2011 at 9:38 pm
Malcolm Gladwell has written on university rankings for The New Yorker. Read the original article and you will laugh at the concept – and people who take it seriously…unless you think that the money-making frauds who design them harm university education…?
http://m.newyorker.com/reporting/2011/02/14/110214fa_fact_gladwell
Dan
April 16, 2011 at 12:35 am
The USA produces some of the most malignant doctors and lawyers from their universities. These places are status mills; they all know how to protect each other, even when criminal events occur. It’s absurd that “reputation” would be one of the ways of determining rank.
April 16, 2011 at 6:52 am
Malcolm Gladwell’s New Yorker article on university rankings systems can be read in full here – it’s both informative and amusing:
Click to access 5695CN_UnvChicagoLawSchool_061111.pdf
April 17, 2011 at 10:15 am
It’s very easy to be diverted to giving attention to these league tables from the more serious challenge of creating a reputation that makes others want to be associated with an institution.