I've been looking for a sensible way of scoring multiple-response questions (question where the student may select more than one answer).
An attractive approach is put forward in "CAA - It's a MUGS Game! Does the Mathematics of CAA Matter in the Computer Age?" by Michael McCabe and David Barrett (Portsmouth). Their system gives a mean undeucated guesser's (MUG's) score of 0 and awards partial marks for partial knowledge of an answer. They show that this can be achieved by setting:
- score for each correct choice = number of incorrect choices
- score for each incorrect choice = - (number of correct choices)
They touch on the importance of making the mark scheme clear to the students (I think that lots of practice quizzes, with feedback, are the only realistic way to achieve this).
As they point out, their mark scheme is only valid if you tell the students how many responses are correct for each question. If you simply say "Which of the following..." you just have a series of t/f questions.
They promote the idea of using mathematics to generate other fairer mark systems, for example by using the correlation coefficient to mark ranking questions, but point-out that this would require CAA systems that can handle non-integer grades.
Comments