Saturday, June 04, 2011

The Four Difficulty Levels of Assignments

As a teacher, I have a few different kinds of exercises I like to assign my students.

There's the traditional kind of assignment where there's a definite "right answer" or thing that I'm looking for. Students are at least used to this from other classes, and they are at least easy to grade: just see if the student's answer matches with the answer key. List five causes of the Crash of 83. Define "positive feedback loop" and state whether it emphasizes early-game or late-game strategy. However, I find the scope and usefulness of these to be limited; game design rarely has a single "best way" to do something.

There are assignments where the student has to exercise a specific skill taught in class, but there are many ways to apply it. Write five one-paragraph concepts for potential game projects. Design the combat system for an RTS. Write a new backstory to a traditional strategy game, playtest it and report on the effect of the narrative on the player experience. These mimic real-world design tasks and provide pretty direct training for the role of game designer. Most of my assignments are in this category.

Then there are larger-scale assignments where a student has to combine skills to take on a major project (typically this takes the form of "create a game from scratch"). This is the kind of end-to-end, open-ended design that a student won't get to see again until they're 20 years into their career as a senior designer, or until they go indie.

Lastly, every now and then (no more than once per academic term, maybe as little as once per year), I like to offer an epic challenge: something that, as far as I know, is an unsolved problem of game design. Sometimes this is something I've tried (and failed) to do myself. Sometimes it's just something I'm curious about, but not so curious as to actually take the time to do it myself. Sometimes it's a relatively famous problem that I've seen attempted and failed by many people who are far more brilliant than I am. I'm talking about things like "design a computer role-playing game that plays in 5 minutes" or "design a collectible business card game that can be played with the stacks of cards you get at GDC, with rules simple enough to be printed on the back of your card" or "write a pitch document for a game based on the Shakespeare IP."

Roughly, I suppose you could say these four categories roughly correspond to Easy, Medium, Hard and Legendary difficulty levels.

You might be thinking, "wait - why assign something that's so difficult that I can't even do it myself?" A few reasons:
  • Learning the lesson that Game Design Is Hard. I make every effort to tell my students that game design is not just a matter of sitting down, coming up with "Great Ideas for games" and then sitting back and collecting royalty checks while other people do all the work. Impossible assignments let me show rather than tell and I think the lesson comes across much clearer.
  • Freedom to fail. While I don't come right out and say that any assignment is impossible, I do let my students know when I'm giving them something challenging. They learn that in game design, it is possible to fall flat on your face... and that sometimes it's even desirable to do so, as you can learn a lot from failed experiments. When you already expect your game to suck, you're more willing to take crazy risks in an effort to fail spectacularly.
  • Tiny chance of success. Few design challenges are truly impossible; it's just that the solutions haven't been discovered yet. Try enough times in enough ways and you will eventually reach a solution, if only by accident (the "million monkeys on typewriters" approach, although I'd like to think my students are slightly more skilled than the average monkey). Some day, either by brilliance or blind luck, one of my students might actually solve one of these things. The student that does so is going to have one impressive centerpiece for their portfolio. Even students who don't crack the problem completely but do manage to make a small dent in it, can learn a thing or two about design from what worked and what didn't - and that lesson can be shared with me and the rest of the class.
  • Offering a glimpse of the next level. When teaching new skills, I like to give my students context: not just "here, learn X and Y for this class" but also "here's where you're going to use it later." By giving students some things they can't do yet, I hope to plant a little seed of fascination with a problem that they want to solve, something that makes them want to keep going... something that they can keep coming back to later as they get more knowledge and experience as a way to gauge how much they've grown as designers.


The Best Colleges said...

We wanted to let you know that your blog was included in our list of the top 40 video game design blogs of 2011. Our goal was to highlight blogs that students would find useful and interesting in their exploration of the field. You can view the entire list at

Unknown said...

Hey, I know this is an old post, so apologies for raising the dead.

I've just started teaching, and something I've been trying to figure out is: how exactly do you grade assignments that are as open-ended and focused on creativity as the ones you mention here? What I struggle with is trying to be fair and not-arbitrary, to have a system that isn't entirely subjective.

As you mentioned, traditional/'easy' assignments have a right answer and are easy to grade. Beyond that, I've had some success with application/'medium' assignments (e.g. create a Rube Goldberg machine). However, from there on up things seem to get very subjective. Do you reward someone less or more for creating a simple game and doing it very well, or for aiming high, working hard, and coming up short? What about picking one skill (like 3D modeling and animations) to investigate in depth vs doing a little bit of everything? When designing a game, how do you compare between a student that applies an existing framework (say MDA) well, and a student that tries to come up with an entirely new framework?

I know those are a lot of questions, so to kind of sum it all up: when an assignment is creative, open ended, and has a real chance of failure, what advice can you give in putting together a rubric, and then grading it?

Ian Schreiber said...

Great question, Julien. I've seen several approaches.

When I first started teaching, I actually didn't know what I wanted to see, so I kinda left it open - I can tell what "A" work is vs. "B" work, it's usually pretty obvious just from looking, but being able to justify it (or quantify it, or explain it to students) is a bit harder. While I'm sure my students were terrified, I would basically just assign grades and write a nice long set of comments to explain the strong and weak points of the project. This is NOT the best way to do things, because students only end up learning the boundaries and expectations in retrospect.

With some experience, I started creating rubrics. If I'm a good enough game developer to know what a "good" project is, I should be able to say to students what I'm looking for, in one or more general weighted categories. This is generally what I do for most open-ended projects, especially when there are specific skills I want students to demonstrate. For example, in a class where I want them to learn rapid prototyping and iteration, I care less about whether the final game is "fun" then whether there's clear evidence that it has undergone several cycles of playtesting. This is evident by seeing how many really obvious stupid things I run into on initial playtest, the kinds of things that would be caught by ANY playtest group. For a "capstone" project where the students are supposed to actually put out a quality product, I'll be looking more heavily on whether the game is fun, whether the gameplay and UI are polished, whether the art and music styles are consistent and appropriate for the project, and so on.

For classes where the process is more important than the project, having students keep an ongoing design journal or developer diary to document their process can be as large a part of the grade as the project itself. Additionally, if the student has to write a post-mortem where they analyze the strengths and weaknesses of their own project, the things they did right and wrong during their own process, and what they would change as their next steps if the project were to continue - that can also be used to compensate for a weak final project, if the student can demonstrate in writing that they know why their project sucks :). And as a bonus, you can take selections of previous-year student post-mortems and hand them to students in the next year, essentially saying "this is where your colleagues totally screwed up last year, see if you can avoid making the same n00b mistakes." Which they probably won't, but hey, forewarned and forearmed and all that.

Hope that helps!