Here is a note that I sent last year to the IB Online Curriculum Centre, where it generated a certain amount of useful (for me) discussion from a couple of my colleagues at various ends of the earth for whom I have the greatest respect. One of them reminded me that the proposals were badly timed for the TOK review process. But it was a good way to make sure that the proposal didn't have anything unobjectionable in it. I have therefore decided to let things lie till a more propitious time. But I intend to adopt some of my own proposals in starting TOK again next year. I still don't think it's likely to be adapted by the IB as presented. Teachers have become too accustomed and comfortable with the current system to contemplate the changes I have suggested.
This is an invitation to consider a somewhat radical proposal for TOK assessment. It has been lying around in my mind for some time, taking an increasingly concrete shape since the Oxford TOK workshop in 2002, but I was encouraged by the brief discussions in the TOK sessions at the Dresden Conference to give it an airing, for what it is worth.
To those who have collaborated in creating the present assessment system, and those who are quite happy to work within it, my proposal may seem to have ignored the fact that more experienced minds than mine have considered these questions before, and that I have neglected the many good reasons why the assessment has taken its current shape. But my intention is not so much to make a case for replacing the existing mode of assessment, as to explore a possible alternative to the present system by comparing TOK with CAS, its confrere inside the Hexagon.
However, I don't envisage doing away with assessment altogether. The accountability that assessment brings will continue, but the assessment will now be based on portfolios of students accompanied by their self-evaluations, and an evaluation by the teacher. Schools send in a sample of five portfolios selected by the teacher according to quality (ranging from best to worst). Alternatively, as with CAS, the portfolios can be chosen at random by IBCA or by the regional office. The portfolios may contain some minimum stipulated work, with entries that may range from the more traditional journals and essays, through “Socratic dialogues” and plays or film scripts, to presentations on video or flash or Powerpoint or on websites – i.e., the full range of textual representations. They may even involve various forms of imaginative visual or aural presentations. The IB assessors can then judge the performance of the school from the sample.
In the case of schools doing the job properly, assessors might expect the range of portfolios to include some excellent or good ones. What happens with schools which habitually turn in unsatisfactory portfolios (unsatisfactory according to the assessor)? They are sent a warning, for two (or n) years running, then their IB authorisation is withdrawn for persistent failure to match the minimum standards of an IB diploma programme. (‘Persistent’ may be defined as – say - three or more years in a five year span.) The main difference here is that assessors will no more assess the work of students, but evaluate the programmes of schools from the sample portfolios and other supplementary information that schools will be asked to provide. There could even be a roving inspector to check up on schools on a random basis, and provide encouragement and guidance and a frisson of nervousness or – very rarely, I hope - a panic attack!
Why would this be a better alternative?
Firstly, it would focus the minds of both teachers and students away from formal adherence to criteria, to engaging with the spirit of the programme. It is important to remember that what we are assessing in TOK is something as imprecise and elusive as growth in intellectual acumen, critical awareness, and an ability to become aware of basic assumptions and value positions, one’s own as well as others’, just as through CAS, students learn to develop personal growth and self-awareness, and solidarity and empathy with others outside their own social situation. What we are assessing in both TOK and CAS is the kind of quality that experienced teachers and assessors would recognize when they see it, but which not only becomes difficult to measure through criteria, but the very attempt at measuring up to the criteria itself detracts from what the criteria were designed to measure (Goodhart's Law? Heisenberg's Uncertainty Principle for the human sciences?) I would like to see the criteria being transformed into representative descriptions of what good TOK programmes might contain, similar to the ones found in the CAS Guide.
Secondly, this mode of assessment would enable the students to deploy their talents for self-expression to their own best advantage.
Thirdly, by allowing for a wider spectrum of cultures to reflect their own voices, it would partly meet the objections of those of us who suspect that there is a western bias in the content, approach and performance required to demonstrate achievement of the subject goals. (To what extent is this really a problem? Teachers with more experience of teaching TOK in more “deeply” non-western contexts than I have seen may wish to address this question separately.)
Fourthly, insofar as it is formative, the assessment of the portfolio would provide a record of the student’s growth in moving towards the objectives of the TOK programme.
Lastly, it would not disadvantage the schools which are familiar and comfortable with the current assessment style, since they will simply retain the essay and oral presentations as their preferred forms of performance in their portfolios.
Is there any reason to believe that this mode of assessment would be any more imprecise than assessment is in Visual Arts at the moment? Are our musical judgments usually so subjective that we are entirely unable to distinguish good music from bad? Surely we have a body of assessors by now who share a consensus on what might constitute a good TOK Portfolio, similar to the ones that exist between Visual Art examiners for art, and aficionados for music?
To inject a personal note, in the past I have found myself fighting the tendency to teach to the essay topics towards the end of the programme, and I suspect other schools may even be turning their TOK classes into an essay polishing exercise at a roughly equivalent point in their programme. I am seeing instances of thinking critically, considering counterclaims, identifying problems of knowledge, and all the rest of it, in the work of students who - outside the TOK classroom – have difficulty identifying a TOK issue in real life, and sometimes even continue to behave as if they have never heard of TOK in their lives. Although I am confronted every year by evidence of my failure as a teacher of TOK, the odd e-mail from former students describing how TOK fell into place for them in university suggests that I may have done something right after all. I cannot claim to know with any degree of confidence whether the examples of inauthentic TOK performance among my students is evidence that the assessment needs to be changed at all, or indeed whether it needs to be changed in the direction indicated. This is, I suppose, a matter for qualitative research. Perhaps piloting the proposal in a number of different schools, each of which keeps a control group assessed in the current way, may be a way of gaining some empirical purchase on this question beyond the merely anecdotal, although I suspect it may prove to be quite difficult to interpret the results without a firm consensus on their standards of validity. But are we quite certain that the present assessment system achieves what we are looking for?
Let’s give the name "inauthenticity" to the ability to turn out a good TOK essay, yet fail to be tolerant, critical and reflective in real life. I suspect that the present assessment format leaves a good deal of room for inauthenticity. I'm wondering whether it's possible to make TOK more immediately relevant and "authentic" to the lives of students, and judging them by how authentically they themselves are able to internalize the elusive spirit of TOK. I am also asking that schools should be judged by the evidence in their students' work that they are trying to make this possible, just as is done in CAS.