Assessment and Evaluation

People who study education will talk a lot about the differences and similarities between assessment and evaluation. I'll spare you the details, but basically we're dealing with the question: how do we know that learning actually takes place when a student attends a course?

The traditional response is to give the student tasks (often in the form of a final exam or project), and hope that their ability to perform well correlates with their mastery of the subject material. But there's the rub: what we measure (performance on a task) is still different from our goal (learning). There's no way to measure an abstract, transparent concept like "learning" directly, so we have to find indirect ways to take a guess.

It strikes me that there's a parallel here with game development, and metrics-based playtesting in particular. During playtesting, what we'd really like to do is know if the players are having fun, but there's no way to measure "fun" directly. So, we take measurements of things that we hope will correlate: how long did each tester spend on this level, how many players finished the game, how long was the average play session, and so on. But it's still indirect measurement.

In both cases, we still live in an imperfect world. Someone tell me when we solve one problem, because it probably means that we'll have solved the other one as soon as someone puts two and two together.