Monday, May 29, 2006

Can questions grow old?

One of the big issues around the college this week is a news item about our journalism students getting an "old" exam (in Norwegian). The question, as well as the guide for the assessors, has been available online since the question was asked in this form for the first time, in 2003, and it was reused now, in 2006. The students are complaining, they feel they are not being sufficiently respected as the professors are re-using questions.

For this to become such a huge issue is mainly an example of the danger of teaching journalists. In every news-room in Norway there is at least one person who'd like to get back at us - either because they were our students and are frustrated about that for some reason, or because they never got into the study, and are frustrated about that for some reason. Hence, anything that can look like an error becomes a big deal.

The debate on whether to use "old" questions or not is however rather interesting. There is a limited amount of central problems in any limited field of study. This comes from the way exams are set up: they have to relate to what has been taught, what we can expect students to be able to answer within this framework, and what kind of skills we want them to display. This means that already, without using the exact same words as in the question we gave 3 years ago, we tend to recycle problems. "Define integrity in journalism, and discuss the possible flaws in the integrity of the participants in this media debate." Next year it may be "In the two articles which comes with this question, you see a dicussion on integrity in journalism. What are the main issues of the debate, and how do the participants relate to the definition of integrity on your reading lists?" Some new examples, some new wording, same old very important question, if you are discussing journalistic integrity and ethics.

The main problem is that the students also had access to the "correct answer" - the guide given to the assessors. At Volda College some of the staff give out these in order to let the students understand the reasoning behind their grade. This lets the current students understand more about how they could have aswered a question, while new students understand the reasoning behind this type of questions. I am not certain that I agree that these guidelines should be made public, exactly because they may be read as a fasit, the final, authorative way to answer a question. They are not, they are a hint to the assessors about what we expect the students of the class 2003 to be able to write on this subject. The target group for the assessor guidelines is not the students, but the assessors.

But back to the original question: Can questions grow old? I say no. The reason why it appears like they are growing old is the openness practiced in the Norwegian educational system. We give our students access to just about everything around the study, and old exam questions are and have always been seen as an excellent tool for students who want to prepare for tests. By seeing what has been asked in the past, they can prepare for what we may ask this year. When I was a student, back in the stone age before computers, people compared years of question collections, noted which questions showed up frequently and which were never asked, and gauged their chances by these collections. They - we - would read according to complex formulas of chance for being asked one question and not another, a strategic reading which gave a certain direction to an otherwise fairly disorganised student life. Professors knew this, of course, and would plan the questions according to the same lists - and yes, occasionally we'd get the exact same question as somebody had asked 3 - 5- 7 - years earlier.

In other countries I have found that some teachers are shocked by this kind of openness. They hardly talk about their questions to other teachers, much less let the students see what has been used for exams on the subject for the last 10 years. That kind of exclusiveness might have made the students in Volda think we respected their work more. I see this as the opposite of respect. It lets professors re-use the same stack over and over again, holding back a valuable study tool from the students, forcing them to work with maximum insecurity and rely on rumours when it comes to what the professor may ask at the end of term. It also lets professors settle into stale patterns without fear of being officially discovered.

Which will happen in the Norwegian system, if you re-use questions, as we just saw.

I say: That's a good thing. So somebody re-used a question. The PROBLEM the question addresses is as important now as it was in 2003. Only this year, in the assessors guidelines, the old guidelines must be included, with the note: "This is the kind of information students knew about before the exam, and we expect a brilliant student to relate to this and go one step further."

Perhaps that challenge is too much for some?


thomas said...

What I react to is not the reuse of questions. After having studied journalism for three years, I've witnessed the (more or less) exact questions/information given in the first semester resurface in the fourth or the sixth. As they should, because the purpose of discussing those very questions are important in this field. Take ethics, for example: If there's one thing we've learned, it is that difficulties regarding the ethics of journalism never stops to intrigue both journalists, teachers and students.

But the discussion can also be held on a different level: Some students knew that the assessors' guide were out in the open. Some didn't. So, on the day of the exam, when the students were met by the old questions, some of us could relax more with regards to disposition etc., and some couldn't. That's unfair.

One could argue that it's the students' job to figure out what has to be read and rehearsed before an exam. Very well. But if the school really practices the openness you're talking about, then it should announce to ALL the students what information it puts out for us to read. It shouldn't be a matter of cooincidence (like say one student is exceptionally good at digging out rather hidden information in our e-learning system, Classfronter) to get to know the conditions of one's exam.

Torill said...

On principle I agree with you, Thomas - all students should at all times have the same information as all other students.

However: We know that can never happen. So some students were better at using the e-learning system - today, for a journalist, being able to find obscure information in an electronic source is an extremely important skill. I have no problems with some students being rewarded for this skill through better grades. The same goes for libraries - some students are comfortable in libraries and can find interesting but obscure references. Some students are good at talking about the subject at hand and get others to open up and share. Some students even (gasp) talks to the staff, asks questions and shows up for classes.

While I don't like giving out the assessors guidelines at all, since they target assessors and not students (I think the real error was in giving out the wrong kind of information), I am afraid that at one level, education is unfair (very much like life in general). The only consolation is that we do our very best to make it unfair in an impersonal way which depends on the students' skills, not the teachers' preferences. And in this case the information WAS available to all, if they had just spent some time digging through the e-learning system, or had been in a study group with somebody who did and was willing to share.