Posted tagged ‘grade inflation’

Making the grade too easily?

August 21, 2017

It’s mid-summer, and so of course it’s the time of year for breathless comments about grade inflation in universities, and particularly about the number of students being awarded a top grade in their final examinations and assessments. This year again we are told that ‘one third of UK universities and colleges awarded the top grade to at least a quarter of their students.’ Indeed English universities are to expect the British government to initiate ‘a crackdown on the rapidly increasing proportion of top degrees being awarded by universities’.

Grade inflation, in so far as it is an issue, is not restricted to any particular country or system; indeed whatever grade inflation there may be in these islands is not so significant when compared with grade inflation elsewhere. And as it happens, some of the most serious grade inflation, over a protracted period of time going back to the 1940s, has been in the United States, and is continuing into the present time). Indeed this has reached a point where some American educators are pointing out that there is no longer any objective way in which the grades of really excellent students can numerically be distinguished from those who are merely good, because an increasingly large percentage of results is clustered around the top of the range of marks.

In reality this does not particularly tell us that unmerited grades are being awarded, but rather that there may not be an adequate consensus around various pedagogical issues including assessment methods and outputs. Should grades reflect performance, measured as objectively as possible, or should they separate a top-performing elite making up a fixed percentage of students (say, 10 per cent) from everyone else, regardless of the extent to which all these students meet any criteria for excellence?

In the end, the noise in the system around grade inflation may encourage us to ask more significant educational questions about what exactly it is we want a university education to provide and how we want to assess their performance and skills. If that is what we get from all this it will be a good thing. But if we remain stuck in the groove of claims and counter-claims about trends in examination results we are unlikely to address the real pedagogical issues. What we probably need least of all is politicians declaring from outside the system how many students (whose performance they have not seen) merit a top grade.

Advertisements

Not making the grade?

July 17, 2012

It is possible to argue that, whatever those of us in the lecturing profession might think or might like to think, from a student point of view the purpose of participating in a university degree programme is to get the degree – the unit of currency for initial career advancement. In fact, it is not just the degree, but the grade recorded. So for many jobs now, the assumption is that students really need to get a First Class degree if they are to stand any chance of employment in the more sought after jobs.

It is often suggested – and this has been discussed in this blog – that over recent years there has been noticeable grade inflation, with students receiving objectively unmerited marks and with ever larger numbers bunched up near the top of the grade heap. As I have mentioned elsewhere, I am not convinced that this ‘inflation’ is unrelated to performance or merit, but even so it is clear that the spread of marks is not as extensive as it used to be, whatever the reasons. This may prejudice the utility of marks or grades as a tool of differentiation between graduates of different levels of ability.

So is the system of grading no longer useful? Some think so, and most recently Professor Jonathan Wolff of University College London has suggested in the Guardian newspaper that we should give all that up:

‘I’m coming to the conclusion that we should simply issue students with transcripts to record their study, and leave it at that. ‘

Of course there is a whole school of thought that competitive grading of achievement is wrong anyway, and that nobody should be encouraged to think of themselves as more able than anyone else. This is how the issue has been considered in school education:

‘Here are two concrete things teachers can do. First, even if they’re forced to give students a grade at the end of the term, they should avoid putting a number or letter on individual assignments. This helps to make grades as invisible as possible for as long as possible – and therefore minimizes the harm they do when students are thinking about them. Second, teachers can help neutralize the destructive effects of grades – and support students’ autonomy at the time same — by allowing students to participate in deciding what grade they’ll get at the end.’

Seen this way, degrees would become certificates of attendance rather than performance. And as we are moving speedily away from concepts of physical attendance, given the technological alternatives or more generally lower levels of inclination to turn up, they may not be much more than the confirmation that the period of registration for a course has come to an end without the student deliberately dropping out in between. What we need to consider is whether that is sufficient. I’m afraid I don’t think so.

Higher education’s ‘bad ideas’?

November 15, 2011

According to Larry Summers, former President of Harvard University and a senior politician in both Bill Clinton’s and Barack Obama’s administrations, higher education suffers from some ‘bad ideas’. Two of the perhaps surprising ones he lists in an interview with the Washington Post are the end of mandatory retirement in US universities, and small group seminars.

In relation to mandatory retirement, Summers argues that as tenured professors hang on into their old age the average age of the academic staff rises, disconnecting them from the young student body.

The problem with small group teaching, he suggests, is that ‘professors are loathe to give bad grades to students they see at the other side of a table every day.’ In other words, he believes that teaching a small number of students makes it difficult to treat them objectively, and this in turn stokes grade inflation.

Larry Summers is not a typical spokesperson for the academic community, but on the other hand he has a ready audience for his statements. So then, is he right in relation to these points? It has long been my view that the compulsory retirement of academics (and others, for that matter) is now hard to justify. But of course an older average age follows – and does this indeed create an academy to which students will find it hard to relate? And have we been wrong all along to seek to defend small group teaching? Or could it be that better grades flow from the better attention students get, rather than from familiarity?

Guest blog: Has there been a decline in educational standards?

October 2, 2011

Dr Brendan Guilfoyle, Institute of Technology, Tralee

 

Almost two years ago a post on this blog led to the following question: what do we mean by educational standards? While matters have moved on somewhat since that initial debate on grade inflation, it is still worthwhile to attempt to answer this question.

In a recent paper ‘New Metrics for Detecting Changes in Educational Standards, I have considered it from a theoretical point of view and applied the conclusions to updated data from the Irish educational system at second and third level. This is paper 9 of a series published by the Network for Irish Educational Standards investigating this issue in a broader context.

I hope the methods and conclusions are of interest to the academic community and I would like to thank Professor von Prondzynski for affording me the opportunity of this guest post to summarize the results.

The theoretical framework adopted has three fundamental assumptions. These are that:

  1. we are dealing with a mass education system,
  2. the results of assessment provides sufficient grade differentiation,
  3. assessment measures student performance against some hierarchical taxonomy of activity.

The first assumption is to allow for robust statistical analysis, the second means we do not consider pass/fail systems, and the final assumption means that assessment seeks to measure student performance against some framework of activity defining the educational standard. It can be as abstract as Bloom’s famous taxonomy of cognitive activity or it can be more specifically articulated to a particular set of learning objectives.

For our purposes the details of the taxonomy are not important. The key feature is that it is hierarchical: the scale of activity has an ordering that is cumulative. Thus a lower activity must be mastered in order to advance to the next level of activity. Conversely, those who have mastered the higher levels find it easier to perform lower activities. Only a belief in a radical dissociativity of cognitive activity would lead one to reject this assumption for second or third level education, and such a Fordist belief does not appear to be widespread among contemporary educationalists.

Consider then an educational system satisfying our three assumptions: that is, an educational system which produces a grade for a large number of students which measures their performance against a hierarchical taxonomy. This grade could be arrived at in any number of ways and could be, for example, an aggregation of a number of measurements. The resulting grade distribution reflects the attainment of the student cohort against the standard.

From a formal point of view, assessment is then a mapping from an ordered set of abstract attributes to the grade distribution of the student population. Such a grade mapping is determined by numerous interlinked factors, including the nature of the material being assessed, the mode of assessment, the selection of students, as well as historical and institutional factors.

We define declining standards to be a change over time in a given educational system where the grade mapping gives higher grades to those at fixed levels in the taxonomy. Equivalently, decline means that a lower level in the taxonomy is required to attain a fixed grade. In such a situation, the mean of the grade distribution would naturally increase. Reversing the argument does not work directly, however, since grade increase on its own does not necessarily imply declining standards. It could result from a variety of factors, for example better teaching, higher selectivity of students etc.

Indeed, it is precisely this issue that is the main point of contention, if not controversy, in the debate about declining educational standards. When is grade increase a symptom of grade inflation (i.e. declining standards) and when is it a sign of higher student attainment? In the absence of other comparative metrics of performance, how can the former be distinguished from the latter?

In the paper I come to the conclusion that if increasing mean grade is accompanied by decreasing skewness, then we have strong evidence of declining standards. In particular, I consider second and third order effects in the grade distribution and what one expects to see during times of declining standards in mass education systems. It is argued that the hierarchical nature of the taxonomy implies that, during times of declining standards, in general those students operating at a higher level in the taxonomy benefit more. That is, one expects to see a non-linear effect in which the grade distribution, aside from having an increasing mean, becomes more negatively skewed. We should witness a depletion of lower grades and an increase in higher grades as grades migrate across the increasing mean.

Furthermore, an advanced decline in standards leads to a second order effect in the form of decreasing standard deviation. This is an artifact of the ceiling effect whereby the top grades cannot increase any further. Such a decrease in standard deviation undermines the whole ethos of assessment as a measure of achievement in educational settings.

Let us for a moment, consider a simple example that illustrates these concepts. If a third level lecturer gives a hint that a particular topic will appear on the end-of-term examination, this information will benefit a student only to the degree to which they are able to take advantage of it. That is, the best students will pick up on it immediately and make a note, the average student may know that a hint has been given but be unclear as to exactly what it refers to, while the weak student, if they are even present, will have little awareness of what has transpired. Thus the better students are more advantaged than their weaker colleagues and, aside from increasing the mean grade in the class, negative skewness will be introduced into the examination grades.

Should the errant lecturer go so far as to show the students the test beforehand, not only will the mean jump, but most of the students will be squeezed into the top grades and become well-nigh indistinguishable. Except, of course, for the poor student who didn’t turn up that day. In any event, the standard deviation will have decreased.

We then turn to grades for the Leaving Certificate Examination from 1992 to 2009 and undergraduate university awards from 1998 to 2008. From the grade data at both levels one finds that the grade distributions display the characteristic pattern of declining standards: increasing grade mean and decreasing grade skewness.

This is a feature of almost all of the most popular subjects of the Leaving Certificate Examination over the period. Interestingly, mathematics has managed to maintain its standard relative to other subjects by these metrics. Perhaps this is a missing argument about reform at second level: the problem is not that mathematics is too hard, but rather that other subjects have become too easy!

The standard deviation of Leaving Certificate awards has remained more or less stable. Thus while it has certainly slipped down the taxonomy, the examination is still able to distinguish between students in a relative sense.

For Honours Bachelor degrees in all seven universities the mean has increased and the skewness has decreased, again the fingerprint of falling standards. In fact, the universities have seen a dramatic period of inflation (namely 1998-2005) at which point in time the mean has stabilized. Could this be the ceiling effect?

Perhaps more worryingly, university grades are found to have decreasing standard deviation – a hallmark of advanced decline in educational standards whereby assessment fails to distinguish between students. Thus, Irish universities appear to be slouching towards a pass/fail system with relative merit residing with institutional reputation rather than award level.

It is hoped that these findings will move the discussion of educational standards forward. We have presented a theoretical model of educational settings which has predictive qualities, as well as tools for a detailed analysis of grade trends against which to test predictions. These tests track changes over time for a given institution and therefore are not subject to the usual difficulties of comparing across institutions. The available data can be analysed from a number of other interesting perspectives within this framework.

For the Irish education system as a whole, at second and third level in particular, the message is clear: there has been a decline in educational standards over the past 15 years. Those who argue that this is not so – that increasing award levels are attributable to student motivation, improvements in teaching or whatever – must now present their case with both a coherent theoretical framework and empirical data to back up their claims. Otherwise, the debate must move on to how best to halt the decline.

Today’s students

June 28, 2010

I attended two events with several business leaders over the past couple of weeks, and in the course of the discussions on both occasions a number of them expressed the view that recent Irish graduates were not of the same quality and did not demonstrate the same standards as those of previous cohorts a decade or two ago. This view appeared to attract a lot of support, and so if it is held by stakeholders of the university system we may have a serious problem that we need to address.

Two factors appeared to be influencing opinions. One of these was the recent debate on grade inflation; it appears that the allegations made in this context have had some effect in undermining employer confidence in Irish graduates. When I pressed the issue, it seemed to me that the erosion of confidence was not related to any actual negative experience that might be connected with unjustifiably high grades, but was simply a reaction to the allegations made; they were assuming that if this message was being put about it must be true. This demonstrates, to my mind, not only that the debate distorted realities, but also that the university sector was really not good at dealing with it and responding to the points that were made.

The second factor appeared to be a widespread belief that students no longer worked hard at college. The businesspeople I met were largely of the view that students did not apply themselves to their courses as previous generations did, and that as a result they were less well prepared for working life, having got used to a life of idle leisure. I might add that some of those saying this specifically excluded DCU from their analysis, but of course this may have been influenced by my presence.

I was particularly struck by the widespread agreement that this assessment of the quality of our graduates was attracting. I am absolutely of the view that these views are wrong, but I am struck by the fact that we seem to have been unable to make a compelling case, or maybe even make any case at all, for standards in Irish universities. This may also be related to the fact that we are not good at publishing information that would present a more balanced picture, and in particular at getting data that would support our case.

Right now we are allowing it to be suggested to our students and our recent graduates that their achievements are not what they are claimed to be, which for them is a devastating allegation. We owe it to them to establish the real position, and if the criticisms are right we need to correct the problems; but if they are wrong we need to be in a position to establish convincingly that this is so.

‘Grade inflation’ and the Irish Universities Quality Board

March 15, 2010

Yesterday’s Sunday Business Post carried an interview with Padraig Walsh, the chief executive of the Irish Universities Quality Board (IUQB). Over the past fortnight when the ‘grade inflation’ story was doing the rounds, some commentators picked on the IUQB and wondered why a body with a remit to protect quality did not address this alleged decline in standards. In the interview Padraig Walsh makes the point that the IUQB is not a regulatory body and cannot compel the universities to adjust their marking systems.

There may be people who will respond to that by asking what the purpose of the IUQB is if it cannot restrain universities when they are compromising standards. The answer is that there is an important difference between quality and standards: quality assurance is about checking whether decision-making processes are transparent and consistent, whereas standards relate to the substance of the curriculum and its appropriateness. Under law the latter is a matter for each university; changing that would require the state (or some other body designated for this) to implement a centralised curriculum for the entire sector, a move that would be totally incompatible not just with university autonomy but also with international best practice.

Under plans announced some time ago by the government, the IUQB’s functions in relation to quality assurance are to be transferred to a new regulatory agency that will be in charge of quality assurance for the entire post-secondary education sector. There is a risk, I believe, that there will be expectations that this new body will be able to compel individual institutions to revise or change their standards. If this were to happen, the implications will be far more damaging to the reputation of Irish higher education than any perceived grade inflation.

It is worth saying that the IUQB, contrary to the impression given by some commentators, has played an enormously important role in addressing quality issues. Across the university sector it is now accepted without argument that institutions owe a duty to the public to demonstrate that they use methods and apply decisions in a consistent and fair manner, and that publishing quality reviews so that everyone can consult them is in the interests of transparency. These are major advances.

On the other hand, the universities themselves need to explain to the wider public much more convincingly than they have done how they determine standards and how they ensure that these are in line with international benchmarks. Simply declaring that there is no problem will not be sufficiently convincing. There is more work to be done.

The grade inflation show comes to the Dail

March 10, 2010

Last week during parliamentary questions in Dáil Éireann (the lower house of the Irish parliament), the Minister for Education and Science, Batt O’Keeffe TD, was asked to provide information on the outcome of the two investigations he had launched earlier in the week into ‘grade inflation’. Here is what he had to say on third level results.

With regard to higher education, it is the case that the data presented indicate a trend of increasing award levels. The proportion of students gaining first class honours in level 8 degree programmes increased from 11.2% to 16.6% in the institutes of technology between 1998 and 2008, and from 8.3% to 16.2% in the universities between 1997 and 2008. Several contributory factors must be considered, including deliberate decisions on assessment standards prompted by external examiner findings aimed at aligning Irish standards more closely with international norms. Improved and more explicit assessment methods, with the development of learning outcomes-based approaches, and better prepared students are also arguably important factors.

Grade increases in higher education are, however, also argued by some to be indicative of a relaxation of standards. This is a subject of debate across systems internationally. Notwithstanding the inconclusive nature of that debate, my principal concerns in an Irish context are on two fronts. I want to safeguard and enhance the quality of our graduates and to ensure the robustness of our systems of quality assurance. The question of graduate quality is a complex one of fundamental strategic importance. The higher education strategy group is currently addressing the broader challenges involved.

It would have to be acknowledged that this statement is a a more reasoned and thoughtful response than some of the other statements that have been floating around. A couple of minutes later he also made the following statement.

We must not undermine in any way the quality of the degrees awarded by Irish institutions and, therefore, of the graduates they produce. It is because of the flexibility of our graduates and the high quality of the education they have received that multinationals choose to locate here. Continuing announcements of the creation of high-tech jobs indicate that there is confidence in our education system. We must challenge that system into the future. That is why I am putting in place a new qualifications and quality assurance agency which can go into the various institutions and compare one with another.

The first part of that last passage is sensible. The last statement – about using the new combined quality agency to ‘compare the various institutions with one another’ might give more cause for concern, not least because it would appear not to favour diversity of mission within the sector. In the end, for any high quality system of higher education, the autonomy of individual institutions must be respected, and diversity should be encouraged and celebrated. I confess I remain nervous about the future.