Measuring the effectiveness of a Serious Game, Gamification or any other Applied Game must form a critical component of any production. Whether the research is carried out from a scientific perspective, to show financial return on investment or to simply help to identify what worked and didn’t and why.
Unfortunately most developers and even some researchers are not well equipped for considering the key questions that could be asked. And learning the full scope of questions which could be asked is a big part of what’s missing from a developer’s toolkit. This then leads to the same narrowly defined set of questions, such as did they learn what they needed to learn – which seems harmless but has no depth. This is clearly insufficient, as causality is often focused primarily on a few dimensions of the game to the ignorance of all other possible elements.
As a network of excellence in games and learning, GALA has a responsibility to spread scientific excellence in the space of applied games and gamification. And whilst a considerable amount of material has been developed in the past 4 years to facilitate this, the concern for me as part of PlayGen an applied gaming R&D studio, and a GALA partner, is the accessibility and immediate usability of this scientific know-how by our designers, developers and in conversations with clients and partners.
Enter ‘Most Excellent Game’. A toolkit designed to facilitates the process of developing research and evaluation questions for games.
Given that the majority of game developers are not scientifically trained in the art and science of crafting evaluation questions and research hypotheses to pursue when measuring the impact of their creations – a toolkit using a set of cards and game boards provides a simplified method for solving this.
The Cards : There are roughly 60 cards in 5 suites ; Context, Participant, Quality, Game Play and Impact, these are then divided into sub groups including :
Context : The Team, Organisational Policy, Game Domain
Participant : Personality, Professional Involvement, Background, As a Game, As a Learner
Quality : Artefacts, Satisfaction, Learning
Gameplay : Emotional Response, Player Behaviour, Game Elements
Impact : Learning Goals and Learning Outcomes
How it’s played : The ‘game’ is played using a deck of cards on two boards, each board representing one part. Part 1 is about identifying the most relevant elements that are of interest to the developer, researcher, client, end-user – or indeed any stakeholder’s point of view that is deemed the most relevant. Part 2 is concerned with concocting the research hypotheses.
The toolkit builds on the substantial works produced by GALA partners within and outside the project. It will be ready to use shortly, watch this space.