Abstract Details

Shaping Immersive Worlds: Framing Design-based Research As A Methodology For Investigating The Development Of Immersive Virtual Environments For Game-based Learning

Research question:

How can a design-based research (DBR) methodology best inform the development of virtual reality (VR) environments for game-based learning?

Main Issue:

As this paper will demonstrate design-based research (DBR) is an important research methodology for the investigation of the effects of immersive virtual environments on learning. This paper aims to: give an overview of how VR and game-based learning interventions have evolved to date, followed by an explanation of DBR methodology and finally a framework of how DBR has is being utilized in a GaeltechVR: An immersive virtual environment to investigate Irish language learning.

Virtual reality is easier to define by its goal rather than by a description of the qualities it can contain. The aim of virtual reality is to evoke a sense of presence in the user using the immersive aspects of the system(Slater and Wilbur, 1997). Presence is a theoretical concept under much debate in the literature with no one unifying description of its definition. There is consensus that it is a subjective experience linked to ”a sense of being there”. (IJsselsteijn et al., 2000; Riva, Waterworth and Waterworth, 2004; Mikropoulos, 2006; Cummings and Bailenson, 2016)

The main area the virtual reality (VR) research community has been interested in investigating and analysing is the unique aspects of the technology with some promising results into the nature of immersion and presence (Witmer and Singer, 1998; Slater, 2009). In an academic context, this research stretches back to the early 1960s (Freina and OTT, 2015) with a revival of interest in the 1990s (Slater and Wilbur, 1997). The current generation of VR technology has renewed interest and promise in the field. The VIVE and Oculus Rift along with the Microsoft range of VR headsets have revolutionised the cost and the portability of the technology which has led to a third phase of interest in the field. This phase of activity is particularly promising as the software to develop immersive worlds and contexts has become realisable with the low barrier to entry and high performing 3d game engines such as Unity3d and Unreal.

This has led to VR research ongoing in the area of clinical and experimental psychology, for example the investigation into the effects of VR for treating anxiety and phobias (Wrzesien et al., 2015; Wiederhold et al., 2016). Research on the effects of VR for educational purposes is still in its infancy.

Coinciding with these developments in VR is the growth of academic interest in game-based learning. Game based learning research has evolved from humble beginnings as drill and practice “edutainment” (Squire, 2003) in the 1990s. The main aim of which was to use games as a tool to motivate the new generation of digital natives (Prensky, 2003) to a wide-ranging field capable of developing and testing new theories and frameworks for how people learn. Long held theories about learning such as social constructivism (Vygotsky, 1962), individual constructivism (Piaget, 1953) or newer theories like legitimate peripheral participation (Lave and Wenger, 1991) are becoming increasingly influential in game-based learning as the open ended nature of games allow these theories to be researched and tested with a new degree of rigor. These complex, multi-faceted games and theories usually include complex systems along with community and societal aspects to learning involving multiple variables that traditional experimental design methodologies struggle to contend with.

A lot of the research focus has been on using commercial games such as The Sims (Ranalli, 2008) or Civilisation (Squire, 2004) to illicit learning through carefully tailored lessons in the virtual world with a period of debriefing after the gaming session (Crookall, 2010). This research has been highly influential and necessary to lead our theory development, it’s limited however by the capabilities of the virtual worlds being used to test these theories. Commercial games are built with a different focus than that of game-based learning artefacts. Simplistic explanations of complex information and misinformation has been a common complaint among researchers (De Freitas, 2006). Due to the high cost of the development of commercial quality games with complex systems and believable 3d graphics this area of research serves a very useful purpose in helping to provide theory and proofs of the capabilities of games. Open Sims have been another exciting development within the research community, capable of complex modifications and visualising 3d graphics along with the added capability of creating multi-user environments to allow the measurement of social learning in virtual contexts. Second Life in particular has led to a wealth of research on their capabilities (Baker, Wentz and Woods, 2009). The ability to design natural and specific interactions along with the ability to create realistic 3d environments in these applications is limited however, due to these factors’ researchers have been limited by the artefacts and tools available to research the potential of games as sites of learning. Once again experimental design methodologies that are positivist in nature are limited by their inability to design and cater for the specific groups the design intervention is trying to target. In the commercial world of games small development teams of 1 to 10 people have emerged with low developmental cost ‘indie’ games which have hit mainstream wide appeal. “Braid” a 2008 low budget hit game is usually attributed with the creation of the “indie” scene. The game-based learning environment is yet to find a similar breakthrough with a lack of game-based artefacts designed specifically with learning intentions in mind that have hit mainstream appeal. The challenge here is great; game-based learning not only has to satisfy the usability and expectations created by mainstream games, but they also must demonstrate a clear link and evidence to the new learning possibilities of the medium. The most successful work into the field to date has been done by military organisations looking to create realistic training scenarios. “America’s Army” was developed and used specifically for this purpose (Susi, Johannesson and Backlund, 2007). Many traditional classroom subject areas such as language learning or the sciences lag far behind these developments. Game researchers designing virtual environments for learning must be equipped with a methodology that allows them to work in consultation with their target users to iterate their designed artefacts, changing their design intentions and developing their theory and knowledge through a multitude of research methods to expand the current state of the art in the field. A design-based research (DBR) methodological paradigm satisfies these ambitious goals.

This paper outlines DBR as an emerging methodology for the development of game-based learning environments. While this methodology is relatively new, traced back to the work of Ann Brown (1992) and Alan Collins (1992) it offers a flexible approach to educational research which is required for researchers authoring new environments to fulfil their learning agenda. DBR involves the creation of particular forms of learning and studying those forms of learning within the context defined by the means of supporting them. This designed context is subject to test and revision, and the iterations that result play a role similar to that of variation in an experiment(Barab and Kurt, 2004). The term DBR came into use in 2001, between 2001 and 2010 a total of 1940 papers using the term were published(Orngreen, 2015). While the testing methodologies incorporated under a DBR experiment can vary they have a set of underlying principles behind their utilization:

  1. They are situated in a real educational context (Orngreen, 2015)
  2. They focus on the design and testing of a significant intervention(Anderson and Shattuck, 2012)
  3. They utilize mixed methods as a means of analysing the interventions effects(Zheng, 2015)
  4. They involve multiple iterations: refining their design based off the previous cycle(Abdallah and Wegerif, 2014)
  5. They involve a collaboration between practitioner’s, researchers and participants (Koivisto et al., 2018)
  6. They offer comparisons to action research(Randolph, 2008)
  7. They seek to offer a practical impact on practice(Ruschoff and Ritter, 2001)
  8. They introduce newly found design principles from the research process to advance theory and practice (Koivisto et al., 2018)

This focus on an authentic setting, multiple iterations and mixed methods analysis is vital because of the emergent nature of game-based technologies. The emphasis of the design is not on generating truths across all games but to inform specific interventions to help guide theoretical frameworks (Dawley and Dede, 2014). As HCI researchers have discovered “It will never be possible or desirable to establish an ideal, complete theory of interaction design practice” (Goodman, Stolterman and Wakkary, 2011). A DBR paradigm understands this same underlying assumption from an educational perspective. “River City” is one of the best-known games developed using a DBR methodology. It’s focus was on examining situated learning in a multi-user environment game environment. Their ability to iterate and change their theory and research methods as the cycle unfolded lead them to discovering learning patterns that were “not well captured in traditional pre/post-test measures” (Ketelhut et al., 2007).

GaeltechVR is a VR game being developed by the authors using DBR methodologies. This is the first study of its kind investigating the effects of VR technology for situated Irish language learning. The main aim of virtual reality is for the user to perceive themselves in the virtual environment and draw upon spatial cues in order to suspend belief and accept the imaginary world as reality (Neville, Shelton and McInnis, 2009). This evolves the nature of a DBR experiment as the VR environment becomes the context of the user. The authentic setting of the classroom/the real-world dissolves if the user accepts the VR environment as real. DBR allows us to examine how users interact in this immersive world. Utilizing a mixed methods examination of the context. We seek to create a profile of the language community, examine their in-game experiences and explore their learning outputs after use. Figure 1.1 demonstrates a diagram of the intended methodology which involves:

  1. A profile of the users’ attitudes and motivations towards the language are recorded using language questionnaires
  2. Video audio and analytics of their experience is captured to explore how the context is explored by authentic users
  3. Post-test questionnaires are used to explore usability issues in the design along with closed and open-ended questions to explore the learning that took place
  4. This process is iterated upon, with changes made to the environment based on the data
  5. Focus group and interviews are utilized in the second iteration to uncover explorative learning dynamics along with the other research tools from phase 1 to build a holistic view of the intervention and see how the changes to the context are changing the users learning patterns.



As our technologies have evolved over the last 30 years the nature of game-based learning has also been evolving. Games are now capable of becoming fully realised 3d environments. VR technologies are aiding in this sense of immersion removing players from their real-life context to a new virtual context. This offers unique opportunities for the research community to investigate the learning potential of virtual environments. If we seek to provide authentic virtual environments for learning they must offer an experience which is deemed authentic by its users. Design based research is an emerging methodological toolkit which reflects these needs and demands. Iterative design and an investigation of the changes they cause to participants enriches the design of the virtual world and can be used to investigate that the virtual world is having its intended effect. Over the course of this paper we have demonstrated the need for this kind of methodological toolkit followed by a conceptual framework for its usage and finally an example of this methodology in practice with GaeltechVR.

Back to list of abstracts