Assessing continuous assessment
In many ways, notwithstanding technological advances and social and demographic changes, education is still much the same in 2011 as it was a hundred years ago. Today’s student’s experience, from first entry into school to the final year at university, is not fundamentally different from that of previous generations. However, in higher education there has been one major shift: when I was a law student my degree result was based totally and exclusively on my performance in a number of written end-of-year examinations. Furthermore, these were all closed book exams. How I was marked depended on what I was able to remember from my courses and my analytical ability. Well, if I’m honest, analytical ability wasn’t that significant in the mix of things, and I know for a fact (because the examiner told me) that my inclination to add some critical assessment to my answers was held against me in at least one paper. ‘Better people than you,’ the examiner told me frankly, ‘have passed the laws and written the judgements. Your views on them are not material.’ Indeed.
But that’s not the case any longer, and for the past couple of decades there has been a growth of continuous assessment as part of the examining framework. Nowadays between 20 and 100 per cent of a student’s final result in a module may be based on their performance in projects, essays and exercises carried out as part of a continuous assessment programme throughout the year.
Furthermore, in a number of countries this practice has spread to schools. Increasingly the central or exclusive role of examinations has given way to some project work that is counted for the final results. Plans by the Irish Minister for Education and Skills, Ruairi Quinn TD, to reform secondary education in this way have however run into opposition, particularly from the trade unions. The latter have argued that this is not the time to undertake such reforms (given current budget cuts), or that the reform is misguided anyway. Others have suggested that developing continuous assessment in schools prompts the earlier onset of plagiarism, particularly as sources are freely available online.
At one level it seems to me that it is not the role of the teacher unions to have a veto on education policy reform, though of course they are entitled to defend their members’ material interests. But more generally, examination-only assessment in the education system undermines society’s need for educated citizens with critical and analytical abilities and a capacity for lateral thinking. It is time for a proper combination of memory testing (which is also still relevant) and the encouragement of a more intellectual engagement with the subject matter of the curriculum. It is time for these reforms.
Explore posts in the same categories: educationTags: continuous assessment, examinations, Ruairi Quinn
You can comment below, or link to this permanent URL from your own site.
University Blog 
August 29, 2011 at 1:36 am
I guess the one thing I will grant you is that you said memory testing is still relevant. But let’s not go into that!
You completely neglected to mention that the UK is ovine away from a system of continuous assessment back to terminal examinations. Not only do they see this as the more effective way of examining *mastery* but that it will help narrow the gap between boys and girls that opened up under continuous assessment.
If the goal of continual assessment is to assess mastery of a subject or skill, then that mastery – if it exists – can be tested by a terminal exam i.e. mastery is mastery exclusive of the assessment of it.
I think it is unfair to characterise the Union’s as defending their members ‘material interests’ as if this is all they do. No one should have a veto on education policy, but neither should any one be excluded from it. Education is a society concern and any member or organization in society has the right to contribute to it. The unions’s are defending issues of pay and conditions per se in this debate – though those argument are there. Far more worrying is the notion that teachers will be the public determinants of whether Jonny gets a good Leaving Certificate. There are parts of the country where teachers get beaten up for merely putting some constructive comments on term reports.
I could pick more holes but will leave it at this one. A terminal examination and the development of critical, analytical and lateral thinking capabilities are not mutually exclusive.
August 29, 2011 at 7:13 am
I’m a great critic of the current Leaving Cert, but I also have some concerns about the proposed move to continuous assessment.
One of my major concerns would be around issues of plagiarism. Given the experiences of university lecturers in trying to communicate even the fundamental underlying principles of correct referencing and avoidance of plagiarism to 1st year students, it is clear that there is currently no teaching of this in schools. Now, this could be because the current exam-based Leaving Cert doesn’t appear to require it; but either way, to introduce continuous assessment as a replacement for those exams without also introducing a rigorous training in both the techniques and principles of referencing others’ work (as well as of proper research methods for continuous assessment) would be a disaster.
August 29, 2011 at 8:28 am
I’ve been watching the BBC on the Lords Reform 1911 debates. The Irish civil, public & semi-State service unions have all of the characteristics. Plus ça change, plus c’est la même chose.
August 29, 2011 at 12:39 pm
This move that you describe with some reservation is business as usual in many countries and many disciplines, including my own. The move to project-based learning is suited to achieving good outcomes in many contexts, and doesn’t seem to be bringing down the walls. It’s one thing to require a memory test at the end of a semester, quite another to hope that at the end of the year or even the course the things you might want people to remember are still current.
Jilly, I’m interested in the way in which the risk of plagiarism shapes our decision-making, as it often does here in Australia. It’s also often used as the reason we shouldn’t teach online.
August 29, 2011 at 1:26 pm
I certainly wouldn’t rule out any teaching technique *because* of the risk of plagiarism it might entail, including the introduction of continuous assessment for the Leaving Cert. But with plagiarism as with other consequences of changed methods, it has to be taken into account and strategies have to be built into the new methods/systems to deal with it.
My fear is that, given that Irish schoolchildren aren’t currently being taught to do proper research or referencing of sources, the proposed introduction of continuous assessment could occur without also introducing those features into the classroom; and that would be a plagiarist’s charter.
August 29, 2011 at 1:26 pm
Throughout the course of attaining my university, not all that long ago, I had a fair mix of continuing assessment and final examinations in the sum of my coursework. I have a great memory and did very well when a 100% multiple choice final exam was administered. In my experience these were mostly implemented in large classes with hundreds of students in massive lectures. To be quite frank, assessing the knowledge of this many students in such a short amount of time probably couldn’t have been done any other way, without incurring huge costs. But as my time in post-secondary institutions progressed, my marks were based on course-work (e.g. a weekly lab component) as well as papers, reports and big examinations. I feel strongly that the latter is not only a more representative mix (anybody can cram), but it also encourages and empowers the student to get more out of the course, given that they are required to contribute, interact and ultimately challenge themselves on a consistent and ongoing basis.
August 29, 2011 at 2:23 pm
I think written exams test far momre that ‘memory’, as is commonly stated nowadays. I do think there is room for continuous ‘harassment’, as we like to call it in our college. However, experience shows that assessment of continuous material is much more subjective than end of term exams, and v difficult for externs to gauge in any systematic way. There is a real danger that if examination goes the way of 100% CA, students can end up ‘learning’ almost nothing at all.
For example, I am currently getting my car fixed – I hope it is being serviced by a technician who knows the components without having to consult the manual every five minutes
August 30, 2011 at 2:54 pm
Your mechanic analogy is a useful one in that the good mechanic is not just ‘knowing the component’, although knowledge is clearly important. He is also problem solving, isolating the fault, assessing effective repairs and considering sourcing the most competitively priced parts (or not).
The well rounded and effective mechanic could not be validly assessed using a purely knowledge based or theoretical exam as this approach doesn’t match the skills required to be a ‘good’ mechanic.
In the real world, graduates/students are increasingly moving to jobs which demand a high level of transferable cognitive skills, the ability to analyse, sythisise and problem solve. These abilities are underpinned by ‘what they know’ but they are critical in their own right. End of year exams and written exams still have an important role to play, but surely longer term assessments which address other skills are important too?
August 29, 2011 at 4:02 pm
Continuous assessment is a completely bankrupt form of (non-)assessment, which allows workmanlike students to cling on indefinitely in college courses by sheer attrition. Talented students, and/or those with an enthusiasm for going into real depth in a subject, are denied both a proper, deep, learning experience and a proper, expansive forum for exhibiting their talents. Talent and depth of understanding are never tested over an endless succession of bitty projects and assignments (marked out of 10?), which most students find thoroughly boring, depressing, and one which can be death by a thousand cuts for more than a few. It is in all likelihood the source of most mental illness and distress on campus, where students must remain in a permanent state of assessment-readiness. Where is the time for reflection and digression that is so essential for genuine understanding?
But as always, cui bono? There are those, of course, who benefit. The cretinous, bureaucratic mentality of constantly dotting i’s and crossing t’s, favoured by continuous assessment, enables an army of passionless, professionally inept, “I have no interest, I just did it for a job” types to bleed into our professional workforce. In doing so, they disappoint our multinationals, such as Intel, who are forever complaining about the quality of graduate coming through. Why wouldn’t they? When most of what comes out of technical degrees, especially, would be more suited to a permanent post in a dole office (well behind the hatches).
If colleges want to retain quality as opposed to generating talented drop-outs and underachievers, then they should give students enough rope to pull themselves up or to hang themselves. But I suspect continuous assessment is just another trick in the box for dumbing down higher education and providing a nice level of stability and determinism in graduate “throughput”. (You can add relative grading, summer matriculation workarounds and points fixing to the bag too.)
August 29, 2011 at 6:36 pm
Norman, assessment methods do differ across disciplines, but I can assure you that in more than a decade of humanities teaching, I’ve never set a ‘marks out of 10′ assignment or (god forbid) a multiple-choice quiz. What I mean by continuous assessment is, usually, a 3000+ word essay, requiring considerable research, a coherent argument and good writing. How often I get those things is quite another matter (!), but they are well worth students’ time and effort learning to do well, as they promote sustained, analytical thought as well as the ability to write.
I think that a mix of exams and continous assessment, which is what most colleges use, is ideal. I’m still worried that schools won’t teach research and referencing alongside their continuous assessment, however.
August 30, 2011 at 1:34 am
I was just about to respond to Norman to say exactly the same thing! When I used continuous assessment, it was always by individual essay (I have experienced ‘quizzes’, group projects and ‘bite-sized’ discussion posts as a student, and have little time for them as assessment tools), and these essays were a genuine test of students’ ability to reason, argue, assess relevance of material and structure an answer. Length ranged from 2 x 2000-word essays for first years, 2 x 3000-word in the second year, and 5000-word essays as the single assignment for final-year elective modules. And we required full and adequate referencing and a substantial use of varied (and relevant) sources.
Students, in my opinion, learned and retained far more by writing these essays than they would have cramming for a two-hour examination.
My degree was assessed by one terminal examination, which took account of no coursework or examinations I had done in previous years. The final examinations were spread over the period of about ten days. The pressure that put on students was enormous. If a student were ill during that period (whether due to stress or something else), then their entire degree could be adversely affected. If the subjects studied in the final year were just not ones which allowed a student to show their abilities to the fullest extent, their degree was adversely affected. If they messed up on one exam paper, their degree was adversely affected. Education should not be all about what a student is able to do in ten days at the end of their final year, which is why I’m a great supporter of continuous assessment, in combination with some examination components.
August 29, 2011 at 6:08 pm
wow. what a response. I’m surprised that there is such a raw nerve around something that has been done for decades in many countries, but there you go.
In terms of opposition from teaching unions I thought one of the issues, rather than plagiarism, is the lack of willingness to be held responsible for making a judgement on their students since in the usual ‘social networking’ of Ireland there is a likelihood of undue pressure being put on teachers and schools. Memories of letters of support from TDs and priests are still recent for some people.
However, if what we need is a cultural change, then why not? It really is time to be a bit more ambitious and make the changes rather than continue to complain and yet shoot down alternatives.
Whether it can be simply put as continuous assessment vs exams though is not necessarily the best approach to the wider issue of promoting active intellectual engagement and a commitment to persevere when ideas seem difficult.
The other question in the case of Ireland is whether the sheer scale of the junior and leaving certs in terms of the numbers of subjects is sensible or whether more time for a smaller number (though still maintaining a breadth, unlike the overly narrow A level scheme) would be more appropriate.
As for the media frenzy each year – that is another extremely unfortunate situation and contributes greatly to stress and the superficial emphasis on exam tactics, helping to fuel and sustain the ‘grinds’ industry.
August 29, 2011 at 6:34 pm
A continuous assessment component is likely to be implemented at a local school level rather than at a national level. This has a few relevant entailments to consider.
One implication is that each school contains a sample which is representative of the Irish student population as a whole. Because there is a natural expectation that each sample is representative, there will be a natural bias to presume that the marks from locally administered continuous assessments should have a normal distribution touching the full space of possible marks. This means, there is a strong likelihood of some students being given extremely high marks, some students being given extremely low marks, and most students falling in the middle. The bias towards this sort of distribution, locally, will be strong, whether conscious or not.The important thing is that the bias towards having such a distribution will be independent of the standard of the local cohort with respect to the population of students as a whole, for any subject’s examination. If the local cohort is exceptionally good in comparison to the total population, the normal distribution penalizes the students in the middle. If the local cohort is exceptionally bad in comparison with the total population, it provides undue reward to the best of the locally-only high achievers.
This all presumes that all of the locally provided continuous assessment marks are weighted equally with the independently rated nationally administered examination, and not with weightings determined by each school. Another entailment is that local administration of the continuous assessments provides additional scope for schools to manipulate the results. It is intriguing that a primary concern expressed is that the students will each re-discover plagiarism, and not that the schools themselves might have an interest in manipulating the results of annual examinations. Evidence that this is unlikely comes from the report that the teachers’ union wishes to veto the incorporation of continuous assessment, on the grounds that it is an excessive change of work practices. Possibly they have spotted that their material interests are not advanced by eagerly moving to a system in which they may manipulate the distributions to influence which students attain which overall marks.
In both of these entailments, locations where bias can be introduced by individual schools are visible.The way to test whether the bias is real or not is to compare the distribution of locally awarded marks with the distribution of marks accorded in the national exams. If there is a significant difference between the two standards, the local marks can be deemed skewed (but without revealing the true reason for the skew) far more compellingly than the distribution of marks in the national examinations could be. That is, the national examination will always provide the standard that will interest most, and which universities will desire access to, for those students applying to university abroad at any rate (and this whole discussion is predicated on giving Ireland’s young access to the best opportunities possible, whether those opportunities are in Ireland or abroad, it seems safe to assume).
If students are competing with each other nationally or internationally (for places in universities, or other competitions that are based on academic achievement) it makes a lot of sense for the marks to be awarded at a national level rather than by local schools.
My own experience of the risks involved in school participation in evaluations is shaped by the honours art syllabus and examination (back in the 1990s). It might seem like art production (which is what the syllabus comprised, not history of art) could be best served by continuous assessment. In this case one had to produce a portfolio, which was examined in the absence of pupils at the end of the year. Unfortunately, the examiners were in no position to differentiate the submissions which were primarily the work of the students from those which were primarily the work of the art teacher. The art teacher, in fact, produced much of the work for those, believe or not, who were stronger at art than the other pupils, creating an even greater gulf between the likes of them and the likes of me (I have no flair for art production). I still find it odd that art production was a compulsory subject in my school. I would have rathered an additional class in geometry or literature.
However, the argument here isn’t that continuous assessment is useless (on the contrary, continuous feedback is invaluable, and continuous assessment marks are a very concise means of expressing feedback), rather it is that the merits of incorporating the results of continuous assessment marks into final marks for results that are meant to be interpreted at a national level are sparse. National examinations can be revised, along with the curricula, to provide a more stimulating and useful education at the secondary level without introducing spurious confounding factors into the results.
August 29, 2011 at 9:06 pm
Fantastic thread
Perhaps there is scope for a online component or resource site for whatever development does take place
October 23, 2013 at 12:36 am
And the evidence from peer-reviewed research says what? No one here has cited a single scientific study. This is very worrying. Educational pedagogy lost in the wilderness – as usual.