Videogame Assessment Tools Hit Their Mark
Wed Sep 10 2008
As the NASA-funded component of the NASA-sponsored Classroom of the Future's serious games research draws to a close, pilot study and preliminary analyses have shown that the embedded assessment tools the Classroom of the Future™ produced for NASA eEducation do, indeed, measure players' learning and perceived experience.
The Classroom of the Future created the Selene videogame in 2007 as part of an investigation into how students can best learn NASA science through videogames.
Analyses used data from the phase one collection, May 16, 2007, through Aug. 1, 2007. These data represent the gameplay of 96 participants located online throughout the country. The research study used a randomized-groups design with repeated measures. This allowed researchers to manipulate player experience to study changes in players' learning and perceived experience. The research environment randomly assigns participants to groups that play Selene. Half of the participants watch round one gameplay. Half play round one. Both groups play Selene round two. Additionally, half of the players watch instructional videos after playing Selene round one. Half watch after completing Selene round two. Researchers used the embedded assessment tools to measure how the sequence and type of activity (playing or watching) affected player perceptions and player learning.
Selene features a timed report embedded assessment tool that measures changes in learning every 10 seconds. Using the timed report, researchers identified when significant learning occurred for Selene players. These results are preliminary. Analyses using the Classroom of the Future tool for perceived experience, the flowometer, are more complete.
The flowometer measures alignment between players' perception of current activity and optimal experience. Optimal experience is known as "flow." Game design theory and practice aim for games that engage players in flow experience. One goal of the Classroom of the Future is to define design principles for flow-inducing instructional games.
Flowometer data identified the segment of the Selene game that inspires the most flow. The flowometer consistently discriminated player activity (playing or watching). Participants consistently reported higher challenge while playing and higher skill while watching. Although players and watchers reported significantly different round one experiences, the groups showed the same round two learning trends. More importantly, across all conditions players demonstrated the same large jump in learning at the same round two Selene game subsegment.
The learning jump concerns the underlying gameplay mechanic. Selene was designed to simulate the fundamental physics and chemistry systems that science has developed to explain the universe. Timed report data show that Selene players progress toward the learning goal more than 75 percent of the time.
The Classroom of the Future game research team calls itself CyGaMEs (Cyberlearning through Game-based, Metaphor Enhanced learning objects). The CyGaMEs approach to instructional game design prepares player with gameplay experience designed to make hard science intuitive. Using their body to interact with the environment, people construct their understanding of the physical world on a very human scale. Most physical processes occur beyond the range of human perception: too large, too small, too fast, too slow to be perceived by the human sensory system. The CyGaMEs approach produces games in which humans interact, on a human scale, with analogs of those processes. This makes learning intuitive when gameplay is followed by instruction.
Selene was produced by the Classroom of the Future as a virtual world laboratory with which to study design, learning, and assessment within virtual worlds. The game introduces participants to the fundamental concepts of planetary geology using the context of Moon formation and evolution. Players form the Earth's Moon and then pepper it with impact craters and flood it with lava flows. Short instructional videos within the environment feature Chuck Wood, an internationally recognized lunar geologist and one of the country's foremost lunar educators as well as director of the Center for Educational Technologies. The Wood segments inspire Selene players to discover science.
