Educational Game Design Model (NMSU Learning Games Lab)

>>Hi. I’m Barbara Chamberlin. I’m with New Mexico State
University’s Learning Games Lab. That’s what I look like. Well, that’s what I
looked like seven years ago when I took the picture. But, you know, if you add
more wrinkles, longer hair and different glasses,
it’s what I look like. So close counts. The Learning Games Lab at NMSU
is not only a physical space where we do research on games,
the games we’re testing. We do a lot of user
testing there. But also research on
other types of games. It’s also the term we use to
talk about our game development. We’ve been developing
educational games for 20 years now. We started doing games for
kiosks and touchscreen, interactive laser disks,
that sort of thing. One of our first distributed
game was Choices in a Pack, a tobacco prevention
game for youth. We went into CD ROMs
for young children. And then of course,
the Internet came. We moved into that space. So now if you fast forward
and take a look at some of our recent things, we
have games for a wide variety of audiences and content. Things for kids, for
adults, for classrooms. For informal or non-formal
learning. Science Pirates is a
good example of that., a game that addresses national
science standards. But really it’s more about
inquiry-based learning and helping learners
understand processes behind experiment design. Other projects that we’re
working on, Ninja Kitchen. You can see some
placeholder art on this. This is in development
as we speak, actually. Ninja Kitchen is a
traditional diner dash game. But in our diner dash game
you have to actually cook meat to the proper temperature
and wash your hands at the right time and keep raw
meat separate from cooked meals. This is because in middle
school our research is showing that this is when you
start learning how to cook. And it’s when you
start preparing food and learning food
safety processes. So we are trying to help kids
learn food safety processes in a really fun way through a
game we think they’re just going to play because it’s
a lot of fun. Treadsylvania is a game we
developed for ATV safety. Or, some of us grew up
calling them four-wheelers. These ATVs can be
dangerous for kids who think they’re just toys. Kids who show up at
a grandparent’s house or a friend’s house. And so they pile kids on and
they don’t wear a helmet. And they go cycling
down the road. And so we wanted
to create a game — we had a very short
turnaround time on this — to help the game player who may
not know that they are going to ride an ATV in a couple
weeks because they don’t know that their friend has one. To reach that player
with information about why you want to be safe. The idea that you have to move
your weight to control an ATV. And if you have a rider behind
you, you can’t move your weight like you’re supposed
to, that sort of thing. And this is Treadsylvania. So, we have a lot of experience and I think do a really
great job of picking content that people don’t
know they don’t know. That maybe people don’t
really want to teach them in traditional settings, like
food safety or ATV safety. That’s not in most
classrooms these days. And putting that
content into a game that makes learning
about it really fun. But, it was really a luxury to start our latest
project, Math Snacks. Because really, everybody
knows math is important. Everybody needs to know math. And we had an opportunity with
Math Snacks to create a series of animations and games that
get behind the conceptual understanding of mathematics. All of the Math Snacks
are available free online. And the animations and games
can be played online, of course, but also can be downloaded
for the iPod, iPhone or for the iPad. Now, a couple of weeks ago I
gave a presentation trying very hard to make games
that don’t stink. This is what, of course, all
of us in game development do. And at the NMSU learning
Games Lab, I said in the presentation I
think there’s really two things we do that are central
to the quality of the games that we produce. The first is user testing. And we do a lot of user
testing and have really refined that process in our
protocols over the years. And that is another
presentation here on YouTube that you can download
if you like. Just search for my name, Barbara
Chamberlin and User Testing and you can find that. And in that presentation, I said
really there’s a second thing, and that is our instructional
design model for games. And that’s really
what I wanted to talk about in this presentation, the
educational game design model that we’ve been refining
for 20 years now. I really believe that if you
have better instructional design at the start you’re going
to have better game design. As I’ve talked to other
developers I’m always curious: How do you do it when you are
creating an educational game? What’s the model? What’s the approach? And I’m hearing a lot of different things
from different folks. And there’s certainly
some similarities. But for us, we also have learned for our educational game
design model what doesn’t work. And I’ve heard this
from some other people, but what hasn’t worked
for us, what doesn’t work for us is first when game makers
serve as educational designers. Now, this isn’t because
game makers, game developers can’t
be educators. In fact, any good game
really is about education. Even if it’s not an educational
or a serious game per say, you are educating your game
player throughout to progress, to learn, to achieve,
to do better. So game makers can be
great about education. But the problem with this is
that game makers, if they’re not up to date on the
content, they might choose to teach math the way
they learned math. And we might have better
ways to teach math now. We have better methods,
better research on that, better approaches. And so it’s important to have
educational designers in on that who can lend best practices
information, best strategies, good examples on what works
and what doesn’t work. Something else that we don’t
believe works, that doesn’t work for us, is when educators
make their own games. And again, it’s not because
educators can’t think intelligently about games. And it doesn’t mean
they’re not game players. But there is value to hiring
professional animators and professional developers and professional
programmers, and educators. There’s value to them
knowing their content. And for an educator to have to
be wrestling with getting fonts to appear correctly and plus
it just may not be the best use of their time. We’ve found that that’s really
better to give to a professional who spends all their time really
thinking and refining that. And so kind of the
blending of that then, that we also don’t think works,
is when educators design, or instructional designers
kind of come up with a model and then just hand that
off to the developers for the developers to make. And I’ve seen other people
use this approach well. But I’ll tell you why
that doesn’t work for us. For us what we’ve
found is we have yet to develop a game that’s
exactly like we think it’s going to be when we start
developing our game. Games go through
so many changes. In graphics, in approach, in
usability, and the whole thing. And what we found is it’s
really valuable for everybody on the team to know both the
education and the game design so that everybody
can be informed when they go to make changes. And in the situation where the
educators design, they give it to the developers, the
developers might be, “Oh, I don’t think that’s
going to be a fun game, but that’s what they
said they wanted.” And then the educators
look at it and say, “Huh. That doesn’t look really fun, but I guess that’s the best
the game developers can do.” It just doesn’t really extract
the best from both worlds. So, for our educational game
design model, I think the brunt of what makes it
different is really in the beginning early
phases of game design. We certainly start with
broad educational goals. Now, I know this is
sort of, “Well yeah, obviously everyone starts
with broad educational goals.” But I want to talk about what
broad educational goals mean. And I’m going to come back
with some examples of that. You also are going to start
with any of the following. Hopefully not all. But you should start with
your audience, possibly. If you know their age, info
about their existing knowledge. You want to know, for
example, the use environment. Where would this be used? At home? In a classroom? On a mobile device? On a school bus or while
waiting for your brother to take a piano lesson? Who would use it? What would the seat time be? How long? What kind
of a platform? Certainly, a budget,
not only for development but for the timeline for that. Now, you probably don’t
want to have all of these because that restricts
you too much. But if you can start with
many of these things defined, then through discussion you can
further refine the remainder of those things. Let me give you some examples. For our ATV game, we
knew the broad goal was to teach ATV safety
to teenagers. We teased it out more
with our clients. Realized they were really
trying to target kids 8 to 18. That’s a wide development range. And as we teased it
out more and more found that they really
wanted to reach kids that probably had
never ridden before and don’t actually own an ATV. But that audience, like
I mentioned, who’s likely to go get on one next weekend
and just don’t know it yet. They also felt that this would
get played by kids at home by themselves, probably
on the web. And they were expecting about
a 20 to 30-minute experience and we had six months
for development window. That affected our budget. That affected the scope
of our educational goals. That in some regards
affected the seat time. Another example of
that for Ninja Kitchen for our food safety,
we knew we wanted to do food safety
for mid-schoolers. And we know how to do
that for a wide audience. But that was too broad
for our educational goals. So we actually, our partner
did a year of research and focus group testing
with students to identify what our target
age group already knew about food safety,
what perceptions and misperceptions they had. What we really needed
to address. So when I talk about
broad educational goals, they need to be specific enough. You can start seeing
some of the outcomes and the changes you
want to make. Again, with Ninja
Kitchen, we knew we wanted to target mid-school kids. And we thought they could
probably play it in class. But also they might
do it at home. And on this one we had about
a year and a half to develop. Finally, let’s go
over to Math Snacks. So in math, of course, such a
huge area, we knew we wanted to focus on specifically
one year on multiple representations
of numbers. You could see a number in numeral form or
in a picture form. We knew we wanted to
talk about number line. We knew we wanted to talk
about order of operations. And those were a little more
specific when we started. But again, that was a lot
of research just narrowing that down as well
before we started. We knew we were going to
reach sixth graders lacking a conceptual understanding
of how math works. But we also knew that they might
be looking at these at home with their parents
or their teachers who also would appreciate
more clarity in the concepts being covered. And so the use environment for
this, we of course wanted kids to use it wherever they could. But we knew we were doing
randomized control trials for our evaluation
in classrooms. So we needed to be able
to distribute our games in a way we could do
a large-scale testing in classrooms without
buying equipment. And for this we have
a longer timeline. So, once you have a general idea of these things you
can assemble your team. Here’s what has worked
well for us. All of our teams through the
years always have included an instructional designer. That’s kind of the
project leader. They do signoff. They’re liaison with the client. But really they’re in
charge of making sure that educationally this
covers what it needs to. Then we usually bring
in a content specialist and a content educator. Sometimes that’s
the same person. But if we’re teaching science, it’s nice to have a
science expert as well as a science education expert. On Math Snacks we
have a mathematician. And we also have
a math educator. It’s good to have people who
are very clear on the content, who can make sure our
illustrations, our drawings, our representations are
absolutely accurate. But we also need somebody who
can instruct us in the best way to convey this knowledge and what has worked
and hasn’t worked. We have a production manager who oversees the actual
production in the timeline. Then we have developers,
depending on the project, who controls the
size of that team. Certainly artists,
programmers, sound, animators. And then we like to
also have an evaluator. It helps not only to know how
the game will be evaluated when it’s finished, but evaluators can also help us
think of our educational goals in terms of if the learner
learns what we want them to do, what they should
they be able to do? And really, in an ideal
game, it would be great to give them the chance
to do that or demonstrate that knowledge as
part of the game, not just as a pretest situation. Now, here’s where we —
here’s the significant thing that we do, I believe. As a team, when bring everybody
together, and we usually do this in kind of a one-and-a-half to two-and-a-half day
summit to start the project. First, we engage in immersion. Now, what I mean when I say
that is that our game developers and our education,
our content people who may not have
worked together before, they all really immerse
themselves in the content and
understand it. And they all really
immerse themselves in games, sample games, as a group. They discuss it. So that the animator,
who may not be familiar with how you teach variables in math really sees
some great lessons on how variables are taught. And the math educator, who
may not play a lot games, is shown by the gamers
some of the newer trends, some of the great
trends, some games in math that may be aren’t good and why. Really, for both
sides of that group to become immersed
in the other world. So in ATV, the development
group went out and took an ATV safety
course for a day. And it was a lot of fun. And then as a team, they refine
those broad educational goals. And that’s another presentation
I need to do sometime about the process
of how you do that. I’ll address that
briefly in a minute. Then that team, after they’ve
refined those educational goals and become very clear on
exactly what they’re going to be teaching with the game or
what kind of behavior they want to change with the game, then
the team establishes specifics. All those things that maybe need
to be defined, use environment, budget, timing, gets
all those worked out. And they start working
up the design document where they prioritize content. That means they say, “Okay, of
all the things we’ve discussed, here is what we definitely have
to have covered in the game. Here are some things
we’d like covered in the game if we can fit it in. But if it doesn’t
fit in, that’s okay. And here are some
things we all discussed that we are not covering
in the game.” It’s really good to have
those three things defined. Then you certainly want to
define the use environment, the audience, all the specifics
about that sort of thing. And then anticipated outcomes. When our gamer has
finished the game, how will they demonstrate
the knowledge, or the behavior change needed? And then a sample of what
that game could look like. Now, this is not necessarily
what the game is going to look like, but it’s
important to have a sample because it helps the team
all be on the same page. “Okay, now I see how this
objective can be achieved through a type of game.” Certainly, the game developers
learn the content [inaudible]. And that immersion
sets the stage for the whole design process where everybody’s constantly
looking for better ways to address the content. Everybody’s constantly
looking for games. This refining of
educational goals, it has to be done
collaboratively. Because everybody throughout the
changes that happen in a game, there might be something
that happens in the way you get the feedback
screen on the third level. But if the animator who’s
handling that has been involved in understanding the goals
for education from the get go, that animator is better
versed to say, “Oh, you know? This is an opportunity where I
could change the way feedback is presented. And it could be done visually
instead of just giving a score.” That’s been crucial
for us in so many ways to have everybody
really well versed in those educational goals. Now, a sample of the
game could look like — and I always use words
“could,” “might be,” “could possibly include”
in our design document — is for our client
especially to do sign off. But it’s really more
about everybody on the team understanding
and being on the same page. Now, I told you I was
going to come back to those pesky educational
objectives. Let me give you some guiding
questions that help us. And this is probably
the brunt of what we do in that original design summit. We ask questions such as, “How is this content
currently taught? And why is it not working?” And it might be that
it’s taught great, but it’s not extensible
to a large audience. And the game helps us do that. Or it might be parts
of it are done well, but the practice
part isn’t done well. Or it could be the
practice part is done well, but the concepts
aren’t done well. So whatever this concept is, we always engage in
a rich discussion. And this is where having
the content specialist and the content educator
really is so valuable. And finding the right
people to work with. What does the learner have to
understand before they get this? It might be if you’re trying to
help kids understand variables that you don’t need a game
that teaches variables if what they’re really
not understanding is order of operations. So I multiply, divide,
add or subtract. So, it’s good to tease that out. But similarly, you should
have some assumptions about what everybody needs to
know before they play the game so you could set up the game that meets their
existing knowledge. What does the learner
already know or believe? That’s especially important if you need the change an
incorrect knowledge base or on misperception. What common mistakes
do learners make? Often, we evaluate in a way where someone can
answer a question. It seems they’ve gotten it. But then let’s say
a math question. So they can answer a multiple
choice question on it. But when we ask them to explain
their answer, we see, oh, they arrived at it in
this case correctly. They got the right answer. But they arrived at it
using the wrong process. Understanding that can also
help you decide what it is you’re teaching. Sometimes what you really need to teach is not what
you set out to teach. And then what will the
learner do, say, or demonstrate if they really get this? Now, this might be end game. It might be they
could answer this sort of question, of course. But it might be they
can think conceptually or describe it in a way. It might be behavior
that we could observe in them outside of the game. We just need to know the goal that we’re working
towards on that. Okay, so then you know,
you’ve got to make the game. That’s again a whole
other presentation. That’s, you know,
a huge part of it. So let me just give you
three guiding questions that we use while we’re
making the game as we go through that multiple
iterative process of making changes and revising. First, are we creating
inquiry-led environments? Or are we just rewarding for
exposure to the information? Now, let me say, you know, I
think sometimes practice games, some people call
them drill and kill, which I’m not sure is fair, because I think there is a
place for practice games. But in my experience, and we’ve
done practice games before, is that those aren’t
as hard for us to make. Those are a little
easier for us to make. And they don’t need the whole
process that we go through. We just need to find
a really engaging way to provide that process. But helping create
an environment where the learner is led
by their own inquiry, that’s kind of our ultimate goal
that we’d like to try to do. And often, practice
can be a part of that. But it’s a good question
to prompt as you’re going through
development. Certainly, does the player
learn the content, or they come to understand that
they didn’t know it? You know, you have to
be careful with that. A player could learn something and if they aren’t inherently
learning the content, they might just be
told the right answer. They may not remember it. They might just say, “Oh, I
guess I didn’t know that.” And they still don’t learn it. So, it’s good to check that
when you’re developing. And then why does the player
want to play this game? Let’s talk about ATV safety. We weren’t targeting players
who wanted to learn ATV safety. We wanted to create a game that
they wanted to play the game because they felt the
right level of challenge, the right level of flow. They could see their
own progress in a game. It was entertaining. And so it’s important
to look at those issues from a game design
standpoint as well. I do want to say that for us, making the game is
an environment of constant formative testing. We test characters. We test paper prototypes. We discuss things with kids. We do little mini math
lessons before we do something to the game. And that really feeds
that process as well while we’re developing. And then, you know, you also
have to finish the game, evaluate the learning, share
the findings and market and distribute the thing. I don’t mean to make
light of these, because these are
large issues as well. And I think, really, that in
the field of serious games, I would have to say
that evaluating learning and marketing and distributing
are two of the areas that we really — that we need
so much more information on. It’s such a young field. We need more, more,
and more information on how we evaluate
meaningful evaluation, good models for doing that. And then I think all of us in
serious games have the challenge of once we make something
spectacular, even if we give it
away for free, sometimes it’s hard
getting eyeballs for that. And I’m hoping that we as a serious games industry
can find way to work together to kind of fuel our
field for that. Now, the interesting thing
is that as I look back over 20 years, now that
I’ve refined this process and I can articulate it,
and I can explain it, I look back on all the different
games we’ve used and I think, “Did we follow this process? Yeah, okay. Did we follow this process? Yeah, okay.” And so I can say now with
confidence as I look back that the scope of our projects
are different and the size of our teams are different. The core model that we use has
really served us very well. So, I wanted to share this in the hopes there might be
something that you could glean out of it that would
help you as well. Now, this presentation was given
at an NSF meeting, a PI meeting for those who are PI’s of
projects that were funded under National Science
Foundation. We have been very fortunate
to be funded by a number of different agencies
that have fueled our game development work. And this is just the
recent presentation that was given there. So, that is my contact
information. Please call me, please email me if you have any questions
for me. Thank you.

1 thought on “Educational Game Design Model (NMSU Learning Games Lab)”

  1. When Guile Hideout makes his appearance she, Brandon, Spenser, and Tucker attempt to fight him but are easily blown away by his sword.


    Also like Generation II, the Generation IV games retain much compatibility with their Generation III counterparts, though in a different manner, and introduce many new Pokémon which are related to those of the previous three generations..

Leave a Reply

Your email address will not be published. Required fields are marked *