INTENTIONAL MUSEUM BLOG

Throwback Thursday: Future Energy Chicago Simulation

The case study below is from a project RK&A did with the Museum of Science and Industry in Chicago, IL and highlights the importance of iterative testing.

Future Energy Chicago Simulation [2013]

An evaluation of a multimedia simulation for a science museum

Between 2012 and 2013, RK&A conducted four rounds of formative evaluation of the Future Energy Chicago simulation for the Museum of Science and Industry in collaboration with the design firm Potion. In the simulation, up to five teams compete against each other in five games: Future House, Future Car, Future Neighborhood, Future Power, and Future Transportation. In the games, players have to make decisions that challenge them to think about energy production and usage, and they receive badges as rewards for selecting energy-efficient choices.

How did we approach this study?

RK&A included primarily middle school youth (home school groups, etc.) in testing, as they are the target audience for Future Energy Chicago. Each round of evaluation explored unique issues relevant to a particular design phase. In the first round of evaluation, RK&A tested three-dimensional paper prototypes of each game to explore middle school youth’s understanding of the concepts presented. In the next two rounds (alpha and alpha prime), RK&A tested the games on touch-screen monitors to explore each game’s functionality as well as youth’s motivations and learning, including a badge system aimed at rewarding youth’s energy-efficient choices. In the last round of evaluation, RK&A tested the games using a combination of multi-touch and projection technology that closely mirrored the final simulation environment. For each round of evaluation, RK&A staff conducted observations and interviews with middle school youth who played the games.

What did we learn?

Each round of evaluation revealed successes and challenges of the Future Energy Chicago games that MSI staff and Potion designers used to improve the games’ functionality and messaging. Throughout testing, findings revealed three key characteristics of the game that were compelling to middle school youth—variety of energy choices, opportunities to design aspects of their energy environment, and challenging energy problems to solve. Findings also revealed that youth’s prior knowledge and experiences with energy choices highly influenced the choices they made and the messages they took away from each game. A consistent challenge throughout testing was helping youth understand the idea of trade-offs in energy choices (comfort or cost versus saving energy). A badge system was implemented to address this issue, as well as to incentivize youth to select energy-efficient choices.

What are the implications?

This study underscores the importance of iterative testing when evaluating a complex digital learning environment. Not only did MSI staff and Potion designers need to understand barriers to effectively using the games, including the intuitiveness of the technology, the Museum needed to understand what about the simulation motivated youth’s game play and effectively empowered them to make smart energy choices as the future residents of Chicago. Further, RK&A facilitated reflective discussions between rounds of testing that enabled MSI staff and designers to apply the study findings and recommendations to the next round of testing, ultimately improving the overall functionality and effectiveness of Future Energy Chicago.

Leave a Reply