We Make Games. Evaluation Of A Game-design Project In Austrian Secondary Schools
The project We Make Games aims at bringing game-design to secondary schools as a means of teaching for subjects others than software engineering and media design. About 50 teachers from 24 Austrian schools (various types of secondary schools from all over Austria) took part in the project which was supported and financed by the Austrian ministry of education. Starting with a train-the-trainer workshop (teaching basics of game-design), the first part of the project concentrated on teacher training and implementing the game-design method in classes. Participating students (taught by the teachers having taken part in the teacher training workshop) were asked to come up with a game design idea for a serious game and present their idea and their team (ideally also a paper prototype) in a short video to be uploaded to the project platform. All entries were evaluated by a jury of game (design) experts. Basically, the students were free to choose their topic as long as it was a serious game. In this context, serious games are defined as being developed for a different purpose than pure entertainment (Michael & Chen, 2005; Bogost, 2007). 78 teams submitted a video and 8 of them were chosen to take part in the second part of the project where students (and teachers) were invited to a two-days kind of game jam where experts supported them in making a playable digital prototype out of their idea.
Using game design as a constructionist learning environment (cf. Kafai, 1994) has often been researched from the pupils’ and students’ point of view and their learning progress. However, there have also been studies looking at pupils and pre-service teachers to find out about the potential of teaching and learning by creating a virtual game learning environment for others (cf. Kafai et al. 1998; Ruggiero & Green, 2017). Compared to digital game-based learning, the creation of games is not as widespread. This is probably due to the fact that designing digital games provides some obstacles as for example the lack of special knowledge in creating games, the high resources regarding hardware and software that are required and finally, the high amount of time that is needed to produce a working digital game.
The focus of the project evaluation was put on the teachers participating in the project to show if the game-design approach would be suitable for various subjects and settings in secondary school. Moreover, the research intended to find out if the project could increase teachers’ media literacy and their understanding of serious games. Finally, it showed which pre-conditions have to be fulfilled to implement the approach successfully.
The following research questions were answered by formative evaluation:
- Which (media) competencies of teachers increased because of the project?
- Which (media) competencies should teachers already have to carry out the project successfully?
- Which general conditions need to be fulfilled?
- Is the approach suitable for different subjects?
- Can the project change the attitude towards serious game (design) positively?
- Which learning processes do students undergo (according to the participating teachers)?
A mixed-methods approach was used to answer these questions. Teachers filled in three online-questionnaires (before the project started and while working with the classes During the unstructured observation took place. After the train-the-trainer workshop in October 2017, paper and pencil questionnaires were used to evaluate the workshop and to find out about participants’ knowledge gain. Additionally, some of the teachers were interviewed using semi-structured interviews to gain more insight into expectations, challenges and results of the project. Workshop II basically was a de-briefing workshop in which the teachers talked about their experience and potentials for improving the project setup. Finally, teachers of the student teams that qualified for the second part of the project were invited to a group discussion.
Generally, evaluation of the first part of the project has shown that teachers think the approach works in various subject and is best-suited for cross-curricular work. The concept of the project (train-the-trainer workshop at the beginning of the project, possibility of asking questions in between and constant support by experts) was seen very positively. What is more, as the teachers were taught the basics of game-design, many of them are willing to do similar projects with other classes, even if there is no official project. This proves to be a big advantage compared to a project setup where external experts teach learners how to design a game. Sustainability and repeatability are thus better guaranteed.
Although such a project is more time-consuming than traditional lessons, teachers appreciated the additional skills their students acquired, like presentation, media-literacy, team-work, problem-solving, thinking out of the box skills and it enhanced their creativity. External motivation because of the game-concept competition was also seen positively, although project set-up should be adapted to include categories for students with different experience levels. All in all, most of the expectations that were ranked rather high before the project were met.
Time pressure proved to be a major challenge in the project. Most of the teachers complained about having not enough time for coaching and the students did not have enough time to work on their concepts as the period between start of the project and deadline of the competition was shorter than two months. However, 78 entries show that it is even possible to work well under time constraint. Some teachers would like to have a project like that as an optional subject to avoid having to cut back on the original syllabus. This solution would also prevent teachers from having to invest much unpaid time for supporting their students.
- Bogost, I. (2007). Persuasive games: The expressive power of videogames. Cambridge, MA: Mit Press.
- Kafai, Y. B. (1994). Minds in Play. Computer Game Design as a Context for Children’s Learning. New York: Routledge.
- Kafai, Y. B., Franke, M., Ching, C. & Shih, J.C. (1998). Game Design as an Interactive Learning Environment for Fostering Students’ and Teachers’ Mathematical Inquiry. International Journal of Computers for Mathematical Learning, 3(2), 149-184.
- Michael, D., & Chen, S. (2005). Serious games: Games that educate, train, and inform. Mason: Course Technology PTR
- Ruggiero, D. & Green, L. (2017). Problem Solving Through Digital Game Design: A Quantitative Content Analysis. Computers in Human Behavior, 73, 28-37.