So here comes the ‘Teaching Excellence Framework’ – do we actually need it?

The UK Government has indicated that it intends to introduce a system for assessing teaching quality of a kind that would be comparable with the Research Excellence Framework (REF). In July the new British Universities Minister, Jo Johnson, set out his agenda in a speech to Universities UK, in which he referred to ‘delivering a teaching excellence framework that creates incentives for universities to devote as much attention to the quality of teaching as fee-paying students and prospective employers have a right to expect’. He added that ‘it is striking that while we have a set of measures to reward high quality research, backed by substantial funding (the Research Excellence Framework), there is nothing equivalent to drive up standards in teaching.’

As a result there is now a process under way in England to design a TEF. Right now it is not yet clear whether Scottish universities should also be part of this framework. Some have argued that, for league table and related reasons, it would be important for Scotland to take part; others have indicated they would prefer Scotland not to join the TEF. But in any case, how sensible is the whole idea?

Even in our age of measuring everything to create raw material for rankings, no attempt had been made to date to develop metrics to rate comparative teaching excellence. In part this reluctance has been driven by recognition of variety – teaching will not always take the same form and pursue the same objectives from institution to institution and from subject to subject. But it is also really unclear as to whether there are objective standards that can be measured. Or rather, where something can be measured (such as student satisfaction) it already is.

The big risk inherent in a TEF is that it will punish innovation. Just as the REF (and its predecessor, the RAE) undermined interdisciplinarity and encouraged competent mediocrity, so a new TEF may persuade academics that sticking with traditional courses and teaching methods is safest. I hope we don’t go there.

 

Advertisement

The value of student feedback

A long time ago, in my student days, I was a student representative for my class. We were law students, in a fairly traditional but open-minded university. Our degree programme was in some ways a curious mixture, with some subjects taught to a very conservative syllabus, while others were innovative and ground-breaking. Anyway, the student representatives from the various years got together and mapped out what we thought might be an improved syllabus. We asked to present this to the School’s academic staff, and we were given an opportunity to do so. A number of changes to the syllabus and course structure followed. Of all the things in which I have participated in academic life, this is one of which I am particularly proud.

However, the success of the initiative depended on our enthusiasm and, crucially, the goodwill of the lecturing staff. There was no routine way of registering our views, and I am not sure that it happened again for quite a while. In fact, as a lecturer my first real experience of student input came in my second job, as a Professor of Law in the University of Hull. Before I joined the Law School it had established a ‘staff-student committee’, with equal representation of both students and faculty, and always chaired by a student (but with the Dean present). The students set the agenda, and so everything was potentially a matter for discussion. The committee’s deliberations regularly led to changes in the programme.

In the mid-1990s the School, in line with emerging quality assurance standards in English higher education, introduced anonymous student feedback at the end of each module. Initially this was done by questionnaires sent by post to each student, but the response rate was poor. So we changed, and used the second half of one of the last lectures to hand out the questionnaires and ask the students to complete them there and then, with the lecturer leaving the room and leaving it to a student to collect the completed forms and hand them in. The quality of this particular kind of student feedback was usually very good, and was influential in programme reviews and design.

When I returned to Ireland in 2000, I confess I was surprised that this had not also become a standard practice there – and I am disappointed that it still isn’t. Some lecturers do organise student feedback, but there is no system-wide management of this, and in many degree programmes students have no opportunity to register their views or suggestions or present an assessment of the value to them of what they have experienced. This gap in practice has been picked up by the Higher Education Strategy Group led by Colin Hunt, and in its report it has recommended that ‘every higher education institution should put in place a comprehensive anonymous student feedback system, coupled with structures to ensure that action is taken promptly in response to student concerns’ (page 61). I am wholly in favour of this recommendation, though I might add that its value goes beyond voicing concerns to suggesting improvements and ideas for reform.

As we have come to emphasise that higher education is not just about active teachers and passive learners, it is important that students have opportunities to shape their learning experience. It is not a question of students taking over course design or assessment, but rather of dialogue with learners from which programme design and delivery can benefit – if it is organised well and carried out effectively. Ireland is seriously behind in this, and it is a matter of some urgency that this recommendation is adopted.

Student choice

When I was an undergraduate law student in Dublin in the 1970s, the content of my degree programme was largely fixed for me, but there were some choices. During my first two years studying law, all my subjects (there were no ‘modules’ then) were compulsory, in part of course reflecting the requirements and demands of the legal professions. During the third year, three subjects were compulsory, and students could choose a fourth from a menu of about seven options. In the fourth and final year one subject was compulsory, and the remaining three were chosen by the student from the same list of seven options. That was it.

In today’s higher education environment the basic structure of degree programmes has, in almost all universities, changed fundamentally with the arrival of modularisation. Students must still, in most institutions, opt for a degree programme in a menu made available to them, but within these programmes they can now expect to be able to make significant choices as to which specific elements (modules) they will take, and in a number of institutions these choices will include some ‘free electives’ that can be taken from outside the subject area they have chosen for their degree. So for example, University College Dublin (UCD) describes this part of the framework as follows:

‘In addition, you generally also have a choice of two ‘elective’ modules (subject to module entry requirements, timetable and availability of places), which can either be taken from within your main subject area to deepen your learning, or from outside it to broaden your learning. The choice is yours.’

Sometimes this level of discretion is not popular with academics, who fear that the selection of modules from outside the key discipline may create difficult complications. Modules made available in other programmes may not be easily understood outside of those programmes, and students taking them in this way may lack necessary background knowledge. There may also be budgetary difficulties as student numbers become hard to predict in individual modules.

Nevertheless, as we increasingly emphasise the significance of interdisciplinarity, modular flexibility may become  more desirable. But how far should it be a free choice, and to what extent should it be constrained or at least guided? The traditional understanding of higher education was heavily focused on education within and for disciplines. Is such intellectual compartmentalisation still possible? Do we have a new pedagogical understanding of the coherent formation of students?

Are we set to downgrade teaching in universities?

The original core mission of universities was to provide students with the best possible learning experience. The main obligation of a professor was to teach, and to do it in the best possible way. Let us leave aside for a moment that most lecturers and professors were never actually trained to teach, it was nevertheless their main vocation. As the university model matured, influenced in particular by the Humboldt model of the university developed in Germany (but of which there aren’t actually that many examples in Germany today), and more recently by the best public and private universities in the United States, research began to take on more and more importance, and also began to be the key driver of funding: globally the best resourced universities have the highest research performance.

So as research became the key differentiator between the very good and the not-quite-so-very-good universities, and as research also provided easy metrics on which to assess performance, it became the major basis on which academic career development was determined. Once you tell a person that they must focus on a particular activity in order to secure promotion, that’s what they focus on.

Now something else has also been added to this scenario. Over recent years, in Ireland at least (but I believe also elsewhere), the unit of resource for teaching has been gradually but very noticeably reduced. What that means is that the funding made available for each student (in Ireland paid by the state) has declined in real terms, and more recently in actual numbers. At exactly the same time, and it has to be said for very good reasons, the government has increased the amount of research funding available. But the result of this combination of measures is that the value of teaching is being seriously undermined.

But this experience is not absolutely unique to us in Ireland. A recent issue of the US Chronicle of Higher Education contained an article suggesting that teaching is becoming less attractive to faculty and does not enjoy a high priority status. Professors feel that their universities are pushing them to put research above teaching in the organisation of their working lives, and moreover don’t feel much pressure on them to teach well, either from their universities or from students.

I have no doubt that we must continue to build up, develop and perfect our research performance as a country; our future depends on it. But we must not do this by downgrading teaching, refusing to recognise or reward it and withdrawing funds from it. Teaching in universities should not be an after-thought, and we need to ensure that universities remain places where the quality of teaching is seen as important. This cannot be achieved if we continue to withdraw resources from teaching and learning while ramping up student numbers. It is time we began to recognise that.

The silent majority?

I was having a conversation recently with an American visiting professor who has just spent the past six months in an Irish university (not DCU). He is about to return home, and he was telling me about his impressions and experiences. Most of what he had to say was wholly positive. He has enjoyed his time in Ireland, and he believes he has experienced universities here that, as he put it, ‘perform miracles with minimal resources’. But one thing he found disappointing: the verbosity (or lack of it) of students.

This has been his second time visiting an Irish university. The last time was in the mid-1990s. Since then, he believes, students have become much less talkative, and also much less speculative and creative in their contributions. Fifteen years ago, he thinks, Irish students were always more than willing to try their hand at conducting an argument – though he concedes that often the arguments were fuelled by enthusiasm rather than in-depth research. This time around, he suggested to me, the research and preparation undertaken by students is much better than it was back then, but their creative originality appears to have suffered. They read everything they are asked to read, and they are extraordinarily good at recalling and expressing the things they have read. But they have become reluctant to critique any of it. When asked to speak, they seem intent on first working out what the lecturer wants to hear before delivering just that. But unless prompted fairly energetically, they prefer not to speak at all – they just write non-stop.

I cannot immediately tell whether his experience is typical, or whether he just landed an unusually reticent class; sometimes a mood can spread across a whole group. But if he is right, we need to address the issue. Higher education is about an exchange of views and the mutual testing, between faculty and students, of ideas. Original and creative thinking, and its expression, must be the cornerstone on which much else is built. As a country we must not allow a culture of passive learning to become the norm. So in the first instance, I am interested in how good a description of today’s students my American colleague’s comments are.

If students have indeed gone silent, they need to be roused.

Higher education: measuring success

On June 11, 2009, the Oireachtas Public Accounts Committee assessed (amongst other things) the allocation of money to and by Science Foundation Ireland. At the end of the session the Comptroller and Auditor General, Mr John Buckley, reflected briefly on how one might assess whether the state is getting an adequate return from the investment in research carried out in universities. This is how he summarised the issue:

The challenge is to manage commercialisation such that there is a return on the State investment. This applies to the university sector where there is a need to ensure the process of protecting intellectual property rights and licensing and so on are in place and exploiting research outputs to ensure they are optimised. Then there is the level of industry or industrial promotion. The challenge here is to ensure there is a pay-off for State assistance and investment in research and development. The point is that this pay-off will be increased to the degree that it is linked with commercialisation and market awareness.

There is very little in that statement to argue with, but it needs to be said that in political debates in particular there is often an impatience with what some might regard as the rather vague metrics offered as justification for research investment. The politician’s instinct is always to look for jobs, and to look for them in the here and now. Having got used to sums invested in the IDA producing employment in a short time frame, they expect university research to do the same. But as I have noted previously in this blog, the economic impact of research is often in the job creation that it encourages third parties to make in the shorter term, and in the commercialisation impact in the much longer term. But it is dangerous to look for a ‘pay-off’ in any direct way in the short term. To that extent, universities should not encourage such misleading analysis by themselves offering the prospects of jobs in larger numbers directly created by university research, as this promotes a misunderstanding of why research should be funded.

But even when we assess university degree programmes, how do we measure success? The number of students admitted into higher education? The number of successful graduates? The number of access students? The ‘value added’ of improving results (which are sometimes taken as evidence of ‘dumbing down’, perhaps perversely)? Student satisfaction? Foreign direct investment in areas where graduates provide skills?

It seems clear to me that we are living in an age where everything that is funded is assessed in terms of whether the funded activities satisfy key performance indicators and can therefore be seen to provide value for money. This is an understandable approach, but its application to higher education is complex. We probably cannot – and probably should not – avoid this movement, but we need to develop a much clearer approach as to what indicates that investment in higher education has provided an appropriate return to the taxpayer. We may want to say that the return is vital but (literally) immeasurable: for example that it is demonstrated in an enlightened, skilled, intelligent, entrepreneurial, cultured and adaptable population. But that may not satisfy the spirit of the age that wants greater accountability in a more direct sense. So as a sector we should lead in developing an understanding of how what we do can be justified in such a spirit. This may now be one of our most urgent tasks.

A question of dates

I believe I am right in saying that, from next September onwards, all Irish universities will have academic years divided into semesters, with Trinity College Dublin the last institution to adopt this format. I might just add in passing that, at least if you’re a pedant like me, you can only have two ‘semesters’ per year, the term having a Latin origin and meaning six months – although of course since nobody’s semester covers six months of teaching, maybe we are all using the wrong term. However, the idea of a ‘third semester’ that is sometimes mooted, it seems to me, is an absurd concept linguistically; and whether it has much pedagogical value is also highly debatable. 

However, though we may all now have semesters, we have an array of different arrangements for these, with some universities starting their first semester in early September or so and finishing it (and all associated examining) by Christmas, and others (including DCU) starting in later September and finishing in late January, with a break for Christmas before the examinations. Leaving aside the question of how we should deal with Christmas and Easter for these purposes, there is the other question of whether one of the argued advantages of semesters – a degree of uniformity in the system with some resulting capacity of transfers and collaborations – is not being delivered.

It is likely that, over the coming year or two, one of the key issues that we will have to consider is how we can pool some resources between institutions in order to offer students programmes that have both intellectual depth and provide access to cognate areas. This may require us to consider whether we can synchronise our dates a little better than we have done so far. And that in turn may prompt a useful debate about what, pedagogically, we are hoping to achieve in how we structure and deliver teaching and learning. Some comments responding to my earlier post on modularisation show that we have a long way to go before we have a shared vision of what is required of universities to deliver ambitious, demanding and stimulating education.

Modular universities

When I was a student – and indeed, when I was first a university lecturer – universities in these islands (and, I believe, in much of the English-speaking world) all had a similar academic year: it was structured into three ‘terms’, each with typically between eight and ten weeks. The basic teaching unit was a year-long course, which would be examined at the end of the academic year, usually in a written examination which alone would account for the marks on which student progression would be decided. Through the 1980s it became more common to allow some non-examination assessment, but on the whole this remained the standard approach.

However, there was always some awareness that in other countries this was not the norm. European universities had an academic year of two semesters – indeed, not really an academic year, since the semester was the basic unit of progression. American universities also had semesters, but arranged slightly differently.

The first university in these islands to adopt a semester-based framework was Stirling, which from its foundation in the 1960s had a different view of how student learning and progression should be arranged. From its opening  in 1967, Stirling offered students modular programmes, under which students had to build up ‘credits’ in order to qualify for graduation, and these credits were awarded for the successful completion of a ‘module’; the modules were the ‘courses’ offered to students, and within certain guidelines students could put together their own menu of modules leading to their degree.

On the whole the Stirling model was frowned upon by most universities, and the suspicion was often voiced that this approach to teaching amounted to a ‘dumbing down’ of university study. It didn’t help that in Britain the other early adopters were polytechnics. However, by the 1990s some universities began to look favourably on modularisation, with the University of East Anglia and Bristol University being among the first. And by the end of the 1990s semesterisation and modularisation had become a tidal wave in Britain, sweeping along the overwhelming majority of institutions. At the time I was a Professor at the University of Hull, and in the mid-1990s we semesterised and modularised. It would be fair to say that there was a lot of scepticism in the academic community, not least because those who pushed for the introduction were sometimes very bad at explaining why it should be done, beyond pointing to the fact that everyone else was doing it.

Indeed that was one of the problems of the 1990s wave of modularisation: because it was so often championed by university senior managements and resisted by many academics, it was often poorly designed, minimalist in intent and reach, and often quite simply the old model squeezed into new units. Very often universities found themselves unable to handle the internal budgetary consequences of real modularisation (where students could make flexible portfolio choices) and so restricted it to such an extent that the real purpose was lost. by the late 1990s in Hull, almost no academic would admit to having been in favour of the introduction of modularisation, and I believe this was not an untypical scenario.

The current decade has seen some new thinking, and in many universities modularisation was re-engineered, this time with the proper pedagogical analysis and therefore with much more dynamic effect. My university now, Dublin City University, also modularised in the 1990s, but only recently have we undertaken a thorough analysis of what a revitalised modular structure could produce, and how it could provide both deep learning and flexible choices. The outcome of this analysis was DCU’s Academic Framework for Innovation, which is gradually being rolled out at this point. Similar reviews and reforms are also being undertaken or contemplated in other universities.

The reason why modularisation became so pervasive but also was initially introduced in such an unsatisfactory manner was perhaps because the academic community was willing to accept that there might be a more innovative way to acquire and assess knowledge, but was given inadequate opportunities to explore that and ensure that any reform reflected the insights gained. When modular degree programmes were introduced in dozens of universities in the 1990s it was an extraordinarily rushed job, and was in the end treated as an exercise in academic organisation rather than an exploration of knowledge and its nature and potential. It was, in my view, the right thing to do, but almost everywhere it was done quite badly. Even now, I would not be certain whether, in many universities, academics would not vote for a return to the old systems of terms.

We cannot really go back, but we must get it right. We must ensure that students have a learning experience that grounds them in the basic knowledge and analysis that they need for the area of study they have chosen, and also that they have the opportunity to make flexible choices that will bring them into direct contact with cognate or relevant disciplines; and we must ensure that we assess their knowledge and achievements in an appropriate manner that is both demanding and open to intellectual innovation. Overall, the academy still has some way to go before we have got this right – though I also believe that, in DCU, we have now made a strong start.

Tales from the classroom

I have mentioned before in this blog that, before I took up my present role, I greatly enjoyed teaching. Though I suppose it is not for me to say so, I believe I was a good teacher. But this did not come to me naturally.

Recently I was going through some old papers, and I came across a set of notes I put together for my very first ever lecture, which I delivered in October 1980. Before that, I had for two years delivered small group tutorials in Cambridge, but in 1980 I was appointed Lecturer in Industrial Relations in Trinity College Dublin, and on that day in October I was to give my first ever lecture. I knew there were to be some 50 or so students in the room, all in their final year. I knew I would not even be the oldest person there, and to be perfectly frank I was terrified. I did not know if I could hold their attention or earn their respect, or indeed whether I would be able to control the proceedings.

In my notes I had written down everything I was intending to say, right down to the introductory remarks and the jokes. I would come in firing on all cylinders, holding (as I planned) the newspaper of that day and pointing out the stories that made the subject we were embarking upon relevant. Thankfully for me, 1980 was still a year of industrial action and unrest, so I knew I’d find the relevant news stories.

The hour came and I embarked upon my ordeal. Actually, no ordeal at all. The students probably took pity on me, and welcomed me generously, and worked with me – and I never looked back. For the years that followed, every time I was due to teach I would feel a strong sense of excitement and enthusiasm. The only time I worried again was when, two years later, I started teaching first year students. There were approximately 250 of these in one lecture theatre, and I knew from colleagues that they were hard to control. One very senior economics professor told me he was terrified every time he entered the theatre with this group. Forgive my smugness, they never bothered me and I found teaching them as exhilarating as any other group.

And yet, how good was I really? In my ten years in Trinity I never used anything that could be called technology to underpin my teaching. I think I may on two occasions have used overhead slides – but just two occasions. And maybe I should be slow to admit this in case anyone from those days decides there’s still time to sue me, but I occasionally threw chalk at students if they were not paying attention.

But at least I regarded teaching as a total pleasure, and my students as my partners. Occasionally I would get very argumentative ones, and that was always a particular joy. These students taught me to communicate in an articulate manner, and much of what I have done since has been the result of that learning process. As was my research, because I would always try out my research ideas with tutorial groups, and often their arguments sharpened my own thinking.

I have enjoyed all aspects of academic life – I used to marvel that a university would actually pay me to do what I so loved doing – but nothing more than the thrill of the classroom when a lecture or other class is going well. It doesn’t always, of course, but when it does there is nothing to beat it. Universities are now complex organisations with a variety of key tasks and objectives; but at the heart of it still is the mission to spread and analyse knowledge in partnership with the students.

Re-imagining university teaching

It is a frequently made comment that while the world around us has changed dramatically over the past 100 years, university teaching has not. You could, so some would say, enter a typical university classroom anywhere in the world and take away more or less the same pedagogical experience you would have had in 1908. I think we need to take such comments with a grain of salt: university classes are now conducted in facilities which, often, are quite different from their Edwardian counterparts, and we have access to (and use) technology that was not even dreamt of back then: bear in mind the ubiquitousness of Powerpoint, for example. And then again, think of all the modularisation and credit transfers and goodness knows what that have transformed the university curriculum.

And yet, there is some truth in the assertion. Behind the façade of innovation lurks a pedagogical dinosaur. The lecturer addressing a mixed-background class attending his module and backing up his or her key arguments with Powerpoint slides may be using new(-ish) technology and may be encountering different assessment methods, but the students will be faced with learning processes that are not that far removed from those of past eras.

I am not suggesting that what we are doing is bad, just that it’s not particularly new. And there is, I think, a curiosity in that: the last 20 years or so have seen an avalanche of new initiatives, new quality monitoring, new technology, new teaching methods, new assessment, new expectations by students; and yet, all this ‘new’ stuff has left only a very small footprint in the history of education, if really any at all. Much of the innovation has been driven by bureaucracy rather than pedagogy, and therefore, perhaps predictably, its intellectual impact has been minor.

But the ingredients are all there for a revolution: we have various pedagogical think tanks, including this significant one in Australia, or this one from Britain. And some individual universities have developed really innovative units to explore the possibility of teaching and learning reform, such as this institute in the University of Hull, or the Teaching and Learning Laboratory in MIT.

I suspect that one of the reasons why the impact of the work of educational researchers and innovators has been so limited is because the regulators seeking reform don’t know what they really want and are overloading the system with procedural requirements that actually inhibit, and occasionally even punish, real intellectual innovation. When that is put alongside the naturally conservative professional instincts of academics, we get nothing very radical in the classroom.

It is therefore time to scale down the procedural quality monitoring a little and to focus much more on learning enhancement and innovation. This requires an open mind on the part of both academics and their leaders and regulators. It is time for that debate to begin properly.