Then there's the question of grading. If I have a programmer, a couple of designers, a producer, an artist, maybe a sound guy, all working on different aspects of the same project... how do I grade them?
So far, I've come up with several possible methods:
1) The "I know it when I see it" method. No quantitative grading at all, I just assign whatever grade I feel the student deserves. While this would probably give the grades that are the most in line with what students have done, it's a terrible stress on the students themselves, and it puts a lot of burden on me to Get It Right. And if a student complains that they got an unfair grade, I have no defense.
2) The project-based method. I grade the final project, not the individual students. Everyone gets exactly the same grade. I don't like this, because it doesn't allow for variation in student abilities.
3) The discipline-based method. I grade the programming on the final project, and anyone who was a Programmer gets that grade. I do the same with design, art, audio and production. Everyone gets graded on their own work... except that you quickly realize that everyone's work affects everyone else. Maybe the game has a brilliant design, but it just can't be implemented by the programmer(s) in the amount of time we have; this is an issue with design and programming and production, so who gets a lower grade?
4) The method I'd like to try is a hybrid between project and discipline. I grade the final project on its various aspects, and each student gets a different weighting of those aspects based on their role:
- Programmers: 40% programming, 5% design, 10% art, 10% audio, 15% production.
- Designers: 15% programming, 40% design, 5% art, 5% audio, 15% production.
- Artists: 5% programming, 10% design, 40% art, 10% audio, 15% production.
- Audio: 10% programming, 5% design, 10% art, 40% audio, 15% production.
- Producers: 15% programming, 10% design, 10% art, 5% audio, 40% production.
- Everyone also has 20% class participation. Class time is roughly the equivalent of team meetings, and not showing up for work is obviously bad.
Where the five categories are roughly defined as:
- Programming: does the game work? How buggy is it?
- Design: is the game fun?
- Art: does the game look good?
- Audio: does the game sound good?
- Production: did the game get finished on time with all of the intended features?
I'm hoping to impress on the students that their work affects the rest of the team, and the rest of the team's work affects them, and that in the industry they're ultimately "graded" on the sales of the final game above all else.
2 comments:
Ian! Long time no speak. I found a link to your blog in an announcement from a former colleague of ours. Though I'm not involved in the game industry anymore (or computer science at all for that matter) I might be able to give you some student perspective on your ideas here.
I think they're very innovative and quite likely to produce good results. The one problem I've always had with group work though is the idea that my grade can be significantly damaged by a single unmotivated leech. I think much of that can be alleviated by installing checkpoints and giving group members a chance to give feedback on group dynamics. There has to be clear penalties associated with negative performance early in the project to encourage consistent work and participation throughout the semester.
Your method is probably closer to the harsh realities of real-world production team evaluation but there are also opportunities there to have team members scolded/spanked/fired by the people in suits before a product goes gold. As a student, you need the same sort of stick to waggle at your teammates along with a carrot that you divvy up at the end.
Glad to see you here Gavin! Hopefully the unnamed former colleague can get you my email address so we can catch up on old times properly.
You make a great point; one person can bring down the whole team to an extent. While this is certainly true in the industry, I should be less harsh in the classroom where everyone's learning this for the first time. In the future I can reduce the effect of others and replace it with a "fudge-factor" that I can apply to people who are strong or weak contributors.
For this first run, the 20% "class participation" can serve for that if I find that it needs to. Although, I'm fortunate to have a class where everyone has multiple talents -- all of the game designers know a little art or programming or production, for example -- so students will have the opportunity to "save" their own grade by contributing in the weaker areas if they need to.
Post a Comment