Friday, March 11, 2011

My Problem With Gamification

This blog post is indicative of the kind being thrown around by so-called "gamification" aficionados. I've seen a number of others along the same lines, but this one is fairly succinct and direct, and I think it can act as a proxy for other similar statements. If you're in too much of a hurry to read it, the practical upshot can be summed up in three points:
  1. The education system in the US is broken.
  2. Grades are an outdated game mechanic. This is part of the problem.
  3. Replacing grades with other extrinsic motivations such as virtual currency is superior and will give students the motivation they need to learn.
Okay, so I have no problem with saying education is broken. It's hard to find anyone who thinks otherwise these days. What about the other two points?

Grades may be part of the problem, but they are not an "outdated game mechanic" because they are not a game mechanic at all. Very often I see rewards classified as "game mechanics" but they are not. The term "game mechanic" has a specific meaning to game designers; roughly speaking, a mechanic is a description of a systemic reaction to an event (such as a player input or a given kind of game state). A reward system that describes the conditions on which a reward will be handed out, and the exact rewards tied to what actions, would be a game mechanic. A grading rubric is a game mechanic. A grade or other reward itself is, in game design terms, a resource or a reward (but not a mechanic). Anyone who is going to speak of something as a game, needs to learn their terminology.

Second, and this is where a lot of "gamification" things fail: extrinsic rewards destroy intrinsic motivation. This has been documented so many times, I'm amazed I even have to say it. You could make a valid argument that by their nature as an extrinsic motivator, grades reduce a student's intrinsic love of learning. But to say that replacing one form of extrinsic motivation (grades) with another (virtual currency) seems flawed in the extreme. They have the same problem! Here's a recent example, where the introduction of 'badges' made students concentrate on earning badges to the detriment of their learning. Well, duh!

You might be thinking: Okay, Ian, if you're so smart, then what is the fix to grades? I would say that we need to do a better job separating the grade (assessment) from the actual learning. Let the reward for learning be the fact that you're learning something awesome and it's giving you new skills and abilities, and you are "leveling up" merely by learning it. I understand we can't do away with assessments entirely, but how about we be clear that they are assessments, not rewards? Young kids start out looking at the world around them with a sense of wonder; in theory, it should not be that hard to simply not get in their way as they enjoy learning stuff.

Look at any expert that is passionate about their field... say, a physicist. Do you think they were motivated to learn physics because they got good grades, or because they thought it was inherently awesome to learn about how the world around them actually works? For that person, this fascination with How Stuff Works is their reward, and as teachers we would do well to find out what makes it so fun for that person to do physics all day, and how we can show our students how awesome that is for them, too. And this is something that the external reward systems propagated by "gamification" systems simply doesn't seem to account for.

Let me be clear. As far as using best practices from the field of game design and applying that to make other tasks more fun and enjoyable, I'm a huge fan of doing this, especially when it comes to teaching. At its best, this is what "gamification" is, and I'm all for it. But all too often I see the term "gamification" used synonymously with "external rewards such as points or virtual goods" and that is something we must all be very careful with, because that may solve some problems in the short term but is probably ineffective or even detrimental to learning in the long term.


skysenshi said...

I didn't realize how huge the problem is (in my country, at least, which has an even worse system than the US) until I resigned from the academe (effective april 16, a week from now) and tried to get back into the game industry. I had forgotten what it's like to be in the real world and it showed during my presentation...

Jason said...

Thanks for taking the time to write this up. I wanted to shed some additional light on the LASD blog post that your referenced.

I spent an hour interviewing the students that wrote it, and it became clear that the wording/implications of that post are not exactly what they seem on the surface. It was written by students who were frustrated by other students were exploiting loop holes in some modules to earn rewards and students going back to earlier modules to chase speed badges when they "should" be doing something harder. Everyone thinks the first thing is a problem, but teachers did not think the second was a problem at all because it wasn't preventing students from completing assigned goals and was giving them a sense of success when they might be stymied by a particularly challenging exercises. In addition, it was also clear from that interview that the vast majority of the rewards in the system students and teachers both liked and thought helped further real learning objectives. We are working to shut down the loops holes that were identified and changing the design of several types of activities to minimize the chance that the same types of loop holes will reappear in future exercises.

It's probably worth my writing a separate blog post about that interview.

The rest of the post is a vast generalization about "what's good" and "what works." I feel like this is the inherent problem with the majority of "gamification" discussions, especially around education. We are working in the real world, and we are measuring the results of that work. And despite your derision for our current approach there's evidence that is doing a lot of the "good stuff" that you mention: making students more curious confident learners, giving them greater mastery over content, increasing the number of students who "enjoy" math. Teachers, who we believe are critical for delivering a complete math curriculum, are leveraging that confidence and skill building to do more interesting exploratory learning in the classroom.

We've got a long way to go, but can't help but be happy with the results we've seen so far.

Kristan J. Wheaton said...

I will admit that I, too, tried to implement game based learning techniques naively. I expressed it differently (games = attention and attention = learning) at first but ultimately I came to a very similar conclusion to yours.

The understanding of intrinsic motivation is key. A student that really enjoys playing China Rail will almost certainly learn a good bit about Chinese geography (or at least the names of Chinese cities). A student who doesn't like China Rail isn't going to learn anything at all.

I have tried to shift my mantra from "making games that teach" to "what can we learn from this game?" In other words, finding great games first and then figuring out how I can match those with points in my lesson plan.


Anonymous said...

As a fellow game design instructor, I've just started reading a book by Dr. Yung Zhao titled, "Catching Up or Leading the Way" in which he challenges the notion that our American education system is "broken." He also stresses that assessments and constant testing doesn't provide an accurate measure of an individual student's success and learning. It's a fairly interesting read for educators, especially those that teach outside the testing subjects such as reading, math, & science (biology, chem or physics).

Zack said...

I deal with the same problem at Full Sail every day. Half of my student email threads are about grades and what they have to do to get a higher one or blaming me for the fact they didn't get a 100. Significantly less than half is about actually making games.

Here are a bunch of students in a program where they supposedly have oodles of intrinsic motivation and yet they spend the whole time hung up on these dang numbers and not making games!

I worry that with rubrics and other automations of grading that we exacerbate the problem because student focus gets latched onto "the system" and what they have to do to beat it instead of on what they are working on.

I don't have a solution.

Purum said...

Hey Ian, good post! I guess what you're asking or suggesting is: get your terms right (agreed), and that not everything can, or more importantly should be gamefied, (again, agreed)!

Though, devil's advocate here: an extreme example
a physicist, who might enjoy the process of solving a problem, but what if there are processes that have NO intrinsic motivators? There are processes SO tedious (as considered by many), that having an extrinsic motivators is a necessity when learning. Maybe an analogue to this is the use of calculators. Long division is not enjoyable for many, I'm not sure that it will ever be "fun", and maybe its only intrinsic motivator is for when their calculator runs out of juice. The calculator itself could be the reward/badge... so that more math could be done; faster too!

Maybe the big problem is that outside a class's context, people instinctively know that grades don't matter... Like they say, the bottom 10% of medicine school graduates are ALSO called doctors. I've never asked a doctor about their cum laud achievements, only their malpractice lawsuits (negachievements?). Within the class context they also use it as a social measurement.

The quest here for designers is to try to create a map of that which is not going to improve with intrinsic motivators (toilet cleanup for janitorial work), and to equally find extrinsic motivators that matter outside of context (a gilded plunger).

Zach, I'm gonna call this the "mulligan". What if you offer students to audit the class before they're allowed to register, I'm their 4rth class, they'd have 3 previous months to complete work that will let them in ... they get the work done in a non fail environment (which is what we want them to understand as the difference between academic umbrellas), and you let them in only if they've done it at least once, and THEN the grade becomes a reward for the interest and performance they've shown within the evaluative time.

Andrew said...

This particular brand of the gamification fad is particularly sad because of how much it really misses the point of why games are fun and why they are engaging; it's because they are inherently fun, because playing them is their own reward.

Games tend to be a lot of work, yet they're still fun: this is what we should be trying to pinpoint if we want to gameify things. But I think that the end result will probably just show some of the things educators have been saying for years, that children should be more involved and free to experiment with the subject matter rather than being lectured at.

Ian Schreiber said...

Purum: great question - what to do with course material that is just not intrinsically fun at all, but rather boring and tedious? I have three answers for you.

First, there is an awful lot of course material that IS intrinsically fun, so even if we just make sure those things STAY fun and leave the rest alone, we are way ahead of where things are in many classrooms right now.

Second, a lot of course material that is boring and tedious, the reason for this is that the material is mechanical and can be automated. Challenge this material - does it really have to be there? Is long division still a useful skill at all these days when everyone has half a dozen calculating devices on their desk? I don't know the answer to that, but it could be that some elements of long division ARE useful for something (in which case, the new skills learned that are relevant to that usefulness can be framed in a fun way because you are using them to solve real, interesting problems). Or it could be that some parts of the curriculum really should be removed.

Lastly, for those few things that really DO just require rote memorization or mechanical practice (but are still deemed important and justifiable), for those things I agree with you that adding an extrinsic-motivation layer of badges and achievements can make such a thing more palatable, and in those rare cases where this is the only option, I admit it is probably better than nothing. However, I would argue that these are much rarer than one might think; if something is useful and worth learning, it is useful because you can actually DO something with it. So instead of skill-and-drill, how about finding ways to get the students to use these skills in real-world applications which are themselves more game-like?

Pi said...

It's hard, if not impossible to remove the reward of a good grade. Give someone a compliment on a substantive aspect of their person. That's what a good grade is: a compliment of someone's intelligence/knowledge. Unless you can remove assessment (even a pass/fail assessment) from the education system, it will always be the extrinsic reward that quashes the intrinsic reward that some students might get from learning.

The best we can hope is to maximize the collateral learning for those pursuing their grade/virtual currency/passing assessment.

Sometimes you have to work inside the limitations of the system. In this case, educators have to work with grades.

Simon said...

I agree with the notion that adding a "points" layer to learning is not the magic bullet we are looking for in education. However, this article challenges the use of the terms "intrinsic" and "extrinsic" motivation in this context. It should be of no surprise really, it seems, that motivation is a very individual thing. I might be motivated by cash rewards while you might find the idea completely distasteful.

There are many ways to engage learners with the learning process that do not involve any magic answers such as designing assessment for learning rather than of learning. It's not rocket science (or game science for that matter!).


Simon said...

Sorry I forgot the link the article!!!


adam.moore said...

If a game system is broken, it isn't going to be fixed by tacking on achievements or badges. This is usually what gamification ends up being because it is a cheap and easy addition... but it doesn't fix the problem. It simply lets us feel like we're making progress toward fixing the problem while putting in the minimum amount of effort.

If we want to use game design to fix education, we need to begin at the beginning. Instead of rearranging the furniture on the deck of the Titanic, we need to design, prototype, and build a better boat starting from the high concept.

pilch said...

The issue here I think is the marked difference between the notion of 'gamifying' anything in order to make it more fun, and thus engaging, and thus have people do better at it. And having a bunch of marketing nuts who see dollar signs above everything, taking a half-baked subset of these principles and attempting to use them (often erroneously) to sell more stuff. I'm all for gamification. And the sooner we dispense with the notion of money in this world the better. It is an out-dated concept.

Makubex said...

Broken education system is due to the difference in what a selfless teacher wants and what the university/college expects.

I'm sure a teacher (at heart) will still be happy if one of his best students drops out and creates a kick-ass start-up with what he learns from the teacher.
For ex, if Steve Jobs said in an interview that teacher-X helped me shape such and such culture, it only adds up X's reputation.

On the contrary, what a college expects is good attendance, good percentage (if not 100) of students graduating regardless of their future.

I would certainly back "Replacing grades with other extrinsic motivations such as virtual currency is superior and will give students the motivation they need to learn", but I never really understand why it needs to be virtual.
Why can't it be real enough? Say for example,
Giving out GDC Passes,
Using faculty links to setup a tour (for selected candidates only) through some major game-dev companies
Helping them fund their 1st game

The virtual currency (say badges) are internal to the class room and have no values in the outside world.

If evolution is right, the only reason why people pursue a career / job / earning / education is to live a decent (subjective) life. "Real" currency makes more sense in this case.

The reason why "money" based approach fails is due to lack of accountability.
For ex, consider you're looking for an investor. The very 1st thing they ask is for your commitment - what is it that you are investing.Even to take home-loans, banks ask you to invest some %age as a commitment.

Ideally, this should be the case in classrooms too.

At least, this is what I would have loved to see from my teachers when I was a student.

Lewis said...

Better late than never, I guess.

Gamification is often an attempt to use the trappings of modern games to reward students for participation, not for accomplishment. Perhaps gamification is another aspect of the change in games themselves, from interest in the journey, the process, to interest only in the destination (implying the reward). Games have changed, in many instances, from consequence-based to reward-based, from player responsibility to earn something, to player reward for participation. "The grind" was introduced into MMOs because it's the easy road to leveling up (after all, a simple bot can do it), but it's not enjoyable activity. So why not use game techniques to try to make work that may not be inherently enjoyable to students, more interesting or "rewarding"?

Zack's comments show how students now focus on the destination, not on the journey.

A couple decades ago I taught a LOT of continuing education classes (four hours a day, five days a week, six weeks), computer literacy and beyond. There were no grades, no graded tests. This makes for relaxed students, for sure, and they can concentrate on what they're trying to learn.

Frequently it's assumed that if continuing ed students didn't want to learn, they wouldn't have signed up for the course. In fact I had lots of civil service and military people who were sent to the course.

Did they learn more, or less, than students would have in a graded environment? I don't know. But they enjoyed it more, and I think enjoyment and learning are coupled positively, though I'm sure it depends partly on the subject. I was teaching very practical skills.

Gamification might have made the learning more fun for some, not for others.