Note: This post was written for the GALA Blog by Ángel Serrano, a researcher from the e-UCM e-Learning research group.
So we found out—and then convinced everyone—that game based learning is great, that exploits the concept of learning-by-doing with lower costs in many cases, that is focused on problem solving, that improves students motivation. In summary, that game based learning has lots of advantages over more traditional approaches. But when we suggest teachers to use serious games to teach their students, the first question that it comes to their minds is: ‘Well, yeah, I like the idea, but… How do I assess this?’
And it is a fair question. Not only because teachers need to give their students a good or a bad mark at the end of the year, but because they really need to know what, how and how many their students actually learned playing the videogame. So, how can we, serious games developers, ease this task to teachers? This question could be answered with a relatively new discipline named Learning Analytics.
Just another data mining process
The idea behind Learning Analytics is actually very simple: we collect data derived from students interaction with on-line educational resources, then we perform a data mining process over these data, and finally we extract a set of conclusions. These conclusions can be about students assessment, comparisons between teaching methods and, in general, information that can be used to improve educational processes as a whole.
This idea isn’t new. Other fields, like Business Intelligence—or even Web Analytics—, have been taking advantage of data mining processes for years to help companies to improve their results. What is new is the domain in which is applied.
Current assessments methods at almost every educational level or field are based on written exams. But nowadays, universities—and even high schools—are using Learning Management System and other on-line educational resources to organize their courses. Students interaction with these resources is generating large amounts of data. We can use these data to learn about how students learn.
So Learning Analytics establishes as its main goal to produce information and concrete actions to improve an educational process at any level—since course level to administrative level—, using students interaction with on-line resources as main source of data. Some general guidelines to lead the process have been set, but the real challenge to accomplish is to establish effective methodologies to analyze the data.
Learning Analytics in serious games
Most of current research is focused on analyzing data generated by LMS, and they obtained interesting results analyzing visits and views of courses resources, forums interactions… However, serious games—and all videogames—are interactive by nature, and this interactivity generates a lot of data that can be analyzed. Data derived from the direct user input but also data coming from all the meaningful game events occurred during the game play.
What can we expect from the analysis of these data? First, we can think of simple things that can be applied to any game mechanics or genre:
- Logging when students starts, quits or finish the game: we can know how many people played the game, how many finished it and how many quit. And using a timestamp in the log, how long they played.
- Logging phase changes: if the game is divided in phases, logging whenever a player accomplished one of them provides information about how the student is distributing her/his game play time within the game. Thus, most consuming time phases can be detected and these times can be compared with designers expectations.
- Logging significant variables values: logging final values for some significant variables, or even their evolution during the game, can established the student performance in the game.
These ideas are simple, easy to apply and very powerful. However, we can also think of more complex data:
- Logging direct user interaction: information extracted from these data mostly relies on the game mechanics, and how these data can be interpreted upon it. Usually, this information can be used to determine if the game mechanics are understood by the students.
- Logging in-game events: each game mechanic has its own in-game events. And each of these events has their own semantic in the game. A life is lost, an item is found… Information extracted from these data is determined by the game semantics.
These data is just an initial step towards a whole Learning Analytics process over serious games data. We need to find out the proper methodologies and analysis to extract the maximum information from these data and other data to come.
A more profound approach to these ideas applied to the eAdventure platform can be found in the paper A framework to improve assessment in educational games.