Saturday, November 11, 2006

Final exam rules are finalized

Just finished writing up the rules for the interactive final. I'll hand them out in class Monday, but you get to see them here first!
  • Students get assigned a unique random number from 1 to 17 (there are 17 students in the class). Randomizing prevents anyone from arguing that I intentionally gave them harder questions, or that they got screwed by sitting next to the wrong people.
  • Starting with #1 and incrementing, each student is asked a question. (Most questions will be about a particular game that I'm demoing at the time). The student gives their best answer. Answers are scored as a normal test question.
  • After the student in the "hotseat" is done, anyone else may raise their hand to elaborate or disagree. If several students want to chime in at once, start at #17 and decrement. (In this way, a particularly bright student who keeps completing everyone else's answers can't keep control of the spotlight -- once they answer out of turn, everyone else gets a shot before they can try again.)
  • For those parts of the answer that the student elaborates on correctly, they split the points with the "hotseat" person, 50/50. In this way, the final is collaborative; even if you don't answer your own question fully, you can reclaim some of those points with help from other students. At the same time, students who answer other people's questions can get above 100% on the final. I expect this to have interesting ramifications on students' study strategies...
  • For those parts of the answer that the student elaborates on incorrectly, they lose a quarter of the full points (i.e. half of what they would have been entitled to). This is to discourage random guessing. Educated guesses are encouraged (I hope), since it's a 2-to-1 ratio of gains to losses.
  • If a student doesn't show up for the final at all, the questions that would have been theirs instead become bonus questions for the group. Everyone gets to answer on an index card, with points given (or taken away) as if they had answered out of turn. Students can choose to only answer partially, or not answer at all.
Time permitting, I'd like to add a "lightning round" at the end, quiz-show style: rapid-fire questions, all with very simple, clear answers, directed at each student in turn, with no answering out of turn. This will be the ideal place for those little tidbits of information that I want everyone to know about the industry, but that can't be demonstrated easily by playing a game.

The final is 2 hours long, which is frighteningly short if I'm asking even just two questions per student (it comes out to about 4 minutes per question on average, including time for me to ask the question, time for the student to answer and time for other students to chime in out of turn). Not sure what I'll do if I run out of time; I'll definitely be practicing everything ahead of time to make sure that I can breeze through my own parts, at least.

So, that's what I've got right now. I'd love to hear your comments.


Adam Yulish said...

I'm completely intrigued -- a collaborative final with some competitive elements. You might want to go ahead and make "fixit" answers from students happen in constant order, too. If you always start at student #17, then student #1 will never get a chance to complete someone else's answer.

For your sanity, have them sit in the numerical order you put them in. :)

Two questions per student makes for very high stakes. if a student blows one question, the best they can hope for is to split the right answer with someone else 50/50, then nail their second question, for a final score of 75/100. The rapid-fire lightning round may be a game/grade saver (or breaker) for some people.

Honestly, I think the design of the final is great. I will say that as an undergrad, the high stakes combined with speaking in front of everyone (even while sitting down in class) would have scared the crap out of me. I'd also argue that speaking in front of people when you're put on the spot is a good skill to have.

This format for the final pretty much has you grading answers on the fly. If you do not make the lightning round a written component (maybe hand out the list of questions and tell people to answer the question number that corresponds with their seat number), then I anticipate you'll have a flood of students mobbing you for confirmations of their final scores.

I'd love to hear how the final goes, and I'd love to see a postmortem on the class itself, if you get the time!


Ian Schreiber said...

This is hard to describe in text, but the idea for the "fixit" answers is that it doesn't reset after each question.

For example, suppose on the first question I get supplemental answers from #13 and #10 (in that order). Then on the next question, the first person eligible to chime in out of turn will be #9 (not #17). So everyone should have equal chance to answer other people's questions.

josh said...

Fascinating, original idea - thanks for sharing! I have two questions: 1. What's an example question, and can you describe the gameplay context the question was asked in? 2. I'm picturing an introverted tech design student being punished for their social awkwardness, clamming up under the pressure of publicly stating in front of his/her peers AND going on their permanent record. You've given this exam at least once, you said elsewhere - how did the introverts do?

Ian Schreiber said...

Josh asks:
"What's an example question, and can you describe the gameplay context the question was asked in?"

Sure. I play the game Columns for the Sega Genesis for a few moments, explaining the basic mechanics of the game. One potential question: what genre is this game, and what mechanics make you say that? Another question: magic gems (which are very helpful) appear more frequently when you're about to lose; is this an example of a feedback loop, and if so, what kind?

Josh also asks:
"You've given this exam at least once, you said elsewhere - how did the introverts do?"

I always give the option for students to take a standard written exam instead. The extreme introverts who would cave under these conditions opt for the written.

For students who are mildly introverted, within five minutes they've forgotten that they're taking an exam :). I put on my best fake game-show host voice, refer to students as "Contestants" and their answering out of turn as "buzzing in". The entire exercise is highly collaborative, because you know that when you're answering someone else's question you're potentially helping their score, so it's hard to feel socially embarassed or awkward when everything you say is for the benefit of someone else, and they're essentially cheering you on. I've never had a student totally freeze up... probably because I keep the tone light, and if it appears I'm not taking things too seriously, it allows everyone else to relax a bit too :)

Gilmoy said...

My HCI class used a Jeopardy format similar to this as practice for the Midterm and Final, but not for the tests themselves. Students thought it was fun, and highly engaging.