Jamming The Assessment: Examining The Viability Of A Game Jam Exercise As An Assessment Tool
- Ms. Hanna-Riikka Roine, Helsinki Collegium for Advanced Studies
- Mr. Mikko Meriläinen, Tampere University
- Mr. Ville Kankainen, Tampere University
While game jams, rapid game co-creation events, have seen increased interest in learning contexts (e.g. Fowler et al. 2016; Meriläinen 2019), their potential is still largely untapped. In this paper, we examine game jams and learning from a different point of view: instead of assessing learning in game jams, we examine game jams as a form of assessment.
Earlier research (see Vos 2015) suggests that when combined with assessment elements, games and simulations can enable teachers to assess more complex skills than traditional tests. This potentially allows teachers to have a more comprehensive understanding of what students know. In these instances, assessment can reveal not only what a student knows but also whether they are able to apply their knowledge in a meaningful way. Drawing from the constructionist learning approach (see Kafai & Burke 2015), this study explores how making games, rather than playing them (cf. Vos 2015), can be used as part of student assessment in higher education. Design as a process requires utilizing the acquired knowledge in finding a solution to ill-defined problems (Cross 2007), which requires students to internalize the knowledge and through this supports learning. Further, applying the knowledge in digital game design has potential for empowering students (Kafai & Burke 2015), which can benefit the learning process. As such, using game jams as an assessment method offers twofold benefits – teacher can evaluate how well students have internalized the course contents by evaluating the design outcomes while students get additional learning benefits from the assessment session.
The study is built around a university course titled “Analysing Storytelling in Digital Media” which was taught during the autumn term 2018 at the University of Helsinki, Finland. The course was part of the Master studies’ programme in English philology, and most of the 21 students attending it were English majors with no previous experience in game studies. The course was designed to provide students with a grasp of tools and methods to study storytelling and narratives in digital media and to enable students to apply them to analyse both the processes of creating and using digital media.
The last item on the course was a game jam using Twine, a game creation application for building branching narrative games, with the idea of giving the students a practical insight into designing and telling a story by means of a digital platform. Furthermore, the students were encouraged to make good use of the ideas and concepts discussed during the course.
The game jam was organised as a part of the group exercises during the course, and the students worked in four five-person groups. The groups made preliminary plans for the games beforehand (e.g. discussing them via email), while the actual jam with face-to-face participation lasted approximately two hours. After participating in the game jam as a part of their groups, the students were instructed to write an entry into their course journal and reflect on their personal experiences of the jam. The data for our study comprises of these course journal entries (18 entries of the jam participants; 3 entries of independent experiments from students who could not participate – they tried out making a small game on their own).
Methodology and data
In this qualitative study, we examined the use of game jamming as a pragmatic tool for assessing learning processes and goals of the students. The goals of the “Analysing Storytelling in Digital Media” course were as follows: the students should 1) have a good grasp of the characteristics and affordances of digital media, especially in comparison with the so-called print or legacy media, 2) be able to apply relevant theories to the analysis of storytelling in various forms of digital media (e.g., games, social media), and 3) be able to write reflectively on their own interpretations of the works and applications of the methodology. We looked at these goals in relation the course journal entries written by the game jam participants. In our view, game jams potentially complement the existing, more conventional assessment methods as they strongly encourage the jammers – and, in this case, the students – to apply knowledge they have acquired practically.
Research questions were formulated as follows:
- Is game jamming a viable tool for assessing student learning on a university course?
- What limitations and requirements are there in the use of game jamming in assessment?
We addressed the questions by conducting a thematic analysis as described by Braun & Clarke (2006). Thematic analysis is a qualitative analysis method for identifying themes in a data set. It is useful for its flexibility, as it is compatible with a range of different theoretical and epistemological approaches, but is not limited to any single one (ibid.). This flexibility does not mean, however, that thematic analysis is conducted in a theoretical or epistemological vacuum, as it is conducted following established structure well described by Castleberry & Nolen (2018). As a method it is also closely related to content analysis, and terms are sometimes used interchangeably (Braun et al. 2019).
We identified themes mainly on a semantic level, focusing on what the respondents explicitly reported and sought to interpret it, rather than seeking to discern latent themes in the reports. Our method falls into the school of reflexive thematic coding where the coding process is an organic one and themes result as an output of the analysis (Braun et al. 2019). An inductive approach with a focus on description and interpretation was seen useful when conducting exploratory research on a little known topic where coding framework is difficult to formulate before the analysis process. The use of thematic analysis allowed us to identify themes related to student learning and their experience of the game jam in the data. These themes were then examined against the learning goals of the course to discern how students were applying knowledge and concepts obtained during the course when building a narrative game of their own
Preliminary assessment of the data suggests that the short game jam format encouraged reflection of the course content as well as its application. Results are not uniform, though, with notable individual variation. A short-format game jam as described in this study is therefore likely to work better as supporting other forms of assessment than as an only method. However, results suggest that a more extensive, in-depth game jam could have potential for more a comprehensive assessment.
Additionally, based on the student reports, game jamming is able to lend itself to supporting learning in a higher education setting, provided it is instructed and planned well with a clear structure. Especially in the context of non-game development students, it must be carefully considered whether the learning goals should be focused on school subject content or on the participants’ personal development and heightened learning motivation (see Meriläinen 2019). For most of the participants, this was their first actual experience of game creation, meaning the jam itself served to bridge the gap from theoretical concepts to practice.
- Braun, V. & Clarke, V. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3 (2), 77-101. doi: 10.1191/1478088706qp063oa
- Braun, V. Clarke V. Hayfield, N. & Terry, G. 2019. Thematic Analysis. In Handbook of Research Methods in Health Social Sciences, edited by Liamputtong P. 843-60. Singapore: Springer Singapore. https://doi.org/10.1007/978-981-10-5251-4_103.
- Castleberry, A. & Nolen, A. 2018. Thematic analysis of qualitative research data: Is it as easy as it sounds? Currents in Pharmacy Teaching and Learning 10 (6): 807-15. https://doi.org/10.1016/j.cptl.2018.03.019.
- Cross, N. 2007. Designerly Ways of Knowing. Basel: Birkhauser.
- Fowler, A., Pirker, J., Pollock, I., Paula, B. C., Echeveste, M. E. & Gómez, M. J. 2016. Understanding the benefits of game jams. Exploring the potential for engaging young learners in STEM. In Proceedings of the 2016 ITiCSE Working Group Reports. New York, NY: ACM, 119-135.
- Kafai, Y. B. & Burke, Q. 2015. Constructionist Gaming: Understanding the Benefits of Making Games for Learning. Educational Psychologist 50 (4), 313-334.
- Meriläinen, M. 2019. First-Timer Learning Experiences in Global Game Jam. International Journal of Game-Based Learning 9 (1), 30-41.
- Vos, L. 2015. Simulation games in business and marketing education: How educators assess student learning from simulations. The International Journal of Management Education 13, 57-74.