Abstract Details

A Review Of The Impact Of Auditory Content On Reward Systems In Game-Based Learning

This research reviews how achievement-related auditory-visuo interactions might be used as a robust reward system in games that depend on the player achieving certain learning goals. The significant expansion of the game industry in recent years has allowed for more specialist roles within games development companies. The design, mixing and application of sound is now often covered by a team of sound specialists, affording each individual sound designer more time and resources to focus on detailing auditory interactions within games. The effectiveness of even simple Foley or sound effects in games are in part due to a person’s instinctive ability to connect visual and auditory stimuli [1,2]. The degree of satisfaction that occurs as a result of this multisensory interaction is dependent on how well they fit together [3, 4]. This is something sound designers are well aware of as they spend time using their own perceptual intuition to fuse auditory events with visual objects. While such sound design is not limited to games development (these roles have long-existed in the film industry), the dynamic human-interaction element of such applications makes the audio-visual interactions more engaging [5], and more importantly, more satisfying. Emphasising satisfaction as a reward system based on these interactions can lead to addictive behaviour [6], and is a small part of why unlocking ‘loot boxes’ in mainstream games includes such interactions [7]. The effectiveness of these interactions as such suggests that they can also be used in reward-based learning systems and is the core focus of this research.


  • [1] Shams, L., Kamitani, Y. and Shimojo, S., (2004). Modulations of Visual Perception by Sound. In The Handbook of Multisensory Processes, Ch.2. MIT press.
  • [2] Wada, Y., Kitagawa, N. and Noguchi, K., (2003). Audio-visual integration in temporal perception. In International journal of psychophysiology, 50(1-2), pp.117-124.
  • [3] Spence, C., (2011). Sound Design: using brain science to enhance auditory & multisensory product & brand development. In Audio Branding Academy Yearbook 2010/2011 (pp. 33-51). Nomos Verlagsgesellschaft mbH & Co. KG.
  • [4] Nesbitt, K.V. and Hoskens, I., (2008). Multi-sensory game interface improves player satisfaction but not performance. In Proceedings of the ninth conference on Australasian user interface-Volume 76 (pp. 13-18). Australian Computer Society, Inc.
  • [5] Hunt, A., Hermann, T. and Pauletto, S., (2004) Interacting with sonification systems: closing the loop. In Proceedings. Eighth International Conference on Information Visualisation, 2004. IV 2004. (pp. 879-884). IEEE.
  • [6] Harrigan, K.A., Collins, K., Dixon, M.J. and Fugelsang, J., (2010), May. Addictive gameplay: what casual game designers can learn from slot machine research. In Proceedings of the international academic conference on the future of game design and technology (pp. 127-133). ACM.
  • [7] Drummond, A. and Sauer, J.D., (2018). Video game loot boxes are psychologically akin to gambling. Nature Human Behaviour, 2(8), p.530.

Back to list of abstracts