Abstract Details

Optimising Gamification With Constructive Competition And Video Games

Introduction

This paper describes a mobile gamification platform called Unicraft2, designed to increase the engagement and attainment of undergraduate computing students. Gamification often relies on extrinsic motivators to engage users, e.g. financial rewards, prizes and compulsory participation (Deterding et al., 2011). However, these types of motivational tools have potentially negative side effects, such as reducing intrinsic motivation, increasing stress and damaging peer group relationships (Fuchs et al., 2014). Is there a more positive way to motivate students? If real-world reward and compulsory participation are removed, how will students be motivated to engage fully with gamification projects?

Video games are ubiquitous in society, with commonly recognisable genres, game mechanics and themes. If gamification could capture the look and feel of a videogame then it might be more attractive to students. If it was delivered via a mobile app then students could use it at a time and place of their choosing. If virtual rewards were used, that had no real world value, then they might be less stress-inducing while still perceived as valuable (Behm-Morawitz, 2013). If competition was the key extrinsic motivator, but it was designed to be constructive, then negative side effects might be eliminated (Fulop, 2009). If the mobile app featured embedded learning activities, then students might be more likely to opt-in.

Unicraft2 was designed to resemble a 3D videogame mobile application. It allows students to create anonymous customisable 3D avatars in a fantasy game setting where warriors battle undead monsters. As well as the more commonly used points and leaderboards (Raftopoulos et al., 2015), Unicraft’s battle game mode provides a platform for students to express their academic progress in terms of their power in battle (which is proportional to credits earned). Competition between students is visualised by asynchronous multiplayer battles. Unicraft is used to conduct multi-choice quizzes in lectures, log attendance in tutorials, reward in-class participation and completion of exercises. Rewards take the form of virtual credits, which can be used to buy powerful gear for the student’s avatar.

Major aim and hypotheses

The major aim of this study has been to develop a new theory for the design and implementation of gamification within a higher education setting. Gamification is cheaper and more flexible than educational games, but it traditionally relied on powerful extrinsic motivators that are linked with reduced intrinsic motivation.

  • Hypothesis one: gamification projects that increase the use of game design principles are more fun and engaging

If we capture more of what makes games fun and apply that to real world activities, then those activities are also more likely to be fun.

  • Hypothesis two: when gamification applications look more like videogames, people are more likely to engage

Videogames are ubiquitous in society and if our gamification projects look and feel like a modern videogame then participants find it more attractive and are more likely to engage and then stay engaged.

  • Hypothesis three: educational gamification projects that only use constructive extrinsic motivators are less likely to damage intrinsic motivation

Extrinsic motivators are necessary (points, leaderboards, badges, competition), but can be implemented in a far more “light touch” approach. Constructive forms of competition can be compelling without forcing people to engage. Such an approach is more likely to be seen as positive and fun, with resulting higher attendance, satisfaction and grades.

Design

Unicraft2 builds on the success of Unicraft1 (Featherstone, 2017). There were two main areas identified for improvement:

  • The application should be integrated into the student experience of lectures and tutorials more closely
  • Competition elements should be compulsive to maintain engagement longer

Attendance

Attending lectures and tutorials regularly, plus completing all the work set, has been shown to produce optimal outcomes for students (Paisey * & Paisey, 2004). Within the author’s institution there is primary data supporting this (see Figure 2), the graph shows that students who do not progress from one semester to the next are more likely to have poor attendance.

Students who are attending and completing tutorials tend to be using other learning portals on the list too, but students have problems when they aren’t motivated by any learning portals or focus on one or two that don’t including regular attendance. This gamification project is designed to encourage attendance at lectures and engagement with tutorials, reducing the rate of decline in attendance. Unicraft2’s specific enhancements will also combat the decline in use of the app itself.

Unicraft2

An Android mobile app that students could download and install on their own smartphones or tablets. Initially, motivational content was restricted to points, leader boards, avatar customisation and achievement badges.

After the halfway point in the semester, competitive elements were remotely enabled to attempt to nurture a more constructive form of compulsive competition. Students could also wager some of their credits on the outcome of the quiz answers.

Similar battles could be triggered by staff, allowing the entire cohort to see their avatars fight together.

Methodology

The study was undertaken with a group of first-year undergraduate students studying computing, staff were recruited to administer the study within a programming subject. It took place over one 10-week long semester with 109 students from three different computing courses. Initially, the app had its competitive battle game and wagering system disabled, this was then activated at the halfway point in the semester. Once active, the battle game and wagering system became the main extrinsic motivational levers. To measure student interaction with the app, Unicraft2 logged all student activity to a database. To assess the impact on student motivation a pre-post study questionnaire was used based on the “motivated strategies of learning” (MSFL) survey tool (Artino, 2005). Semi-structured interviews were also conducted after the study to examine student experiences in more depth.

Within the cohort, 54 people used the app every week until the study completed, 15 students agreed to act as a control and not use the app at all. The remaining 40 students dropped out towards the end of the study, this level of drop out was in line with falling attendance trends in this and other first year subjects these students attended.

The app was only offered on the Android platform, due to costs of development, but Android tablets were offered to students who did not have a compatible device.

For the first 5 weeks, the app provided the following gamification functionality:

  • Earn credit for attending tutorials
  • Earn credit for taking part and correctly answering quiz questions in the lecture
  • Earn credit for successfully engaging in tutorials at the discretion of the tutor i.e. answering questions, demoing work, helping other students, completing work outside class, etc.
  • Earn credit for completing tutorial work – this was self-certified by the students due to the time overhead of a member of staff administering it.
  • Create and customise a videogame-like avatar using clothing and equipment bought with the credits earned.
  • View point scores, rank and tutorial progress in comparison with other students using the app.

 

For the last 5 weeks, the app provided the same functionality, plus more explicit video game mechanics and competition.

  • A battle game was activated where the student’s avatars could fight in teams against computer-controlled enemies.
  • During multi-choice quizzes it was possible to wager earned credits on the outcome of the question, potentially doubling the wager (maximum wager was limited to just two credits).
  • Battle games could be initiated:
    • On the student’s device with two peer-matched teammates
    • Autonomously on the class projector with all students in the tutorial participating
    • On a large screen in the cafeteria with all students in the subject group participating.

Results

Unicraft2 recorded all mobile app activity on a cloud-hosted server (see Figure 6), storing anonymised user activity data.

Attendance

All university subjects suffer falling attendance over time, with first-year subjects being particularly problematic. Part of Unicraft2’s functionality involves recording and rewarding attendance. Attendance trends in FOP (Fundamentals of Programming), the subject using the software, outperformed all other first-year subjects.

FOP drew in students from: Computer Science, Computer Science for Games and Software Engineering. These three cohorts attended a number of first year subjects that had their attendance recorded for comparison (See Table 1).

  C.S. C.S.F.G S.E.
Component programming     yes
Fundamentals of programming 2018 yes yes yes
Fundamentals of programming 2017 yes yes yes
Fundamentals of computer architecture yes yes yes
Introduction to game development   yes  
Professional and project development yes yes yes
Algorithms and data structures yes    
Systems modelling   yes yes

All subjects apart from FOP (2017 and 2018) show an accelerating decrease in attendance. FOP (2017 and 2018) is the only module countering this trend and in 2018, when Unicraft2 was used, it had its best year (see Figure 7). Interestingly, FOP 2017 was trialling in-class multi-choice quiz software (Turning technologies, 2015), which shows students are responding positively to more interactive forms of teaching, and this is most noticeable when Unicraft2 was used (an 11% increase).

How usage changed pre and post battle game activation

Just as students lose-interest/tire-of any subject over the semester, they will slowly disengage with gamification if they are not compelled to participate. The constructive competition in Unicraft2 provides extra compulsion. Once the battle game and wagers system activated, app usage rates increased.

Only virtual item purchasing was higher initially, due to students customising their avatars at the start of the study with cheap items.

Impact on motivation

The MSFL questionnaire used pre/post study, measured levels of intrinsic motivation and agency. Extrinsic motivators often cause reductions in these measures (Deci et al., 1999), but in this case (see Table 2), none had a p-value below the statistically significant threshold of 0.006 (Bonferroni correction).

Questionnaire section Variance within experimental group Variance within control group Variance between experimental and control group – pre-test Variance between experimental and control group – post-test
Intrinsic goal orientation 2%

t(58) = 0.62, p=0.5

 

0%

t(19) = -0.06, p=0.96

 

1%

t(28) = 0.33, p=0.7

 

4%

t(14) = 1.07, p=0.3

 

Extrinsic goal orientation 5%

t(58) = 1.1, p=0.3

3%

t(14) = 0.58, p=0.6

 

6%

t(40) = 1.3, p=0.2

 

4%

t(14) = 0.81, p=0.4

 

Task value 0%

t(54) = 0.09, p=0.9

1%

t(16) = 0.24, p=0.8

1%

t(32) = 0.42, p=0.7

3%

t(15) = 0.71, p=0.5

Self-efficacy for learning and performance 3%

t(57) = 0.68, p=0.5

 

5%

t(17) = 1.22, p=0.2

 

1%

t(32) = 0.14, p=0.9

 

1%

t(31) = 0.43, p=0.7

 

Test anxiety 1%

t(45) = 0.13, p=0.9

8%

t(15) = 0.63, p=0.5

0%

t(22) = 0.04, p=0.96

7%

t(13) = 0.62, p=0.6

Control of learning beliefs 1%

t(50) = 0.38, p=0.7

 

3%

t(15) = 1.04, p=0.3

 

4%

t(39) = 1.25, p=0.2

 

8%

t(22) = 2.27, p=0.033

 

Table 2. MSFL questionnaire results, pre and post study

Attainment

Within FOP, marks were compared with the previous student cohorts from 2016 and 2017. The results show an increase in average marks compared to the previous two-year average (see Figure 10). Where this increase is lowest, they were the group that engaged with the app the least.

Course 2016-7 2017-8 Average 2016-8 2018-9 Change
C.S. 63% 67% 65% 69% 4%
C.S.F.G 61% 63% 62% 69% 7%
S.E. 50% 50% 50% 60% 10%

Interviews

The interviews confirmed what could be seen in the app metrics, attendance, attainment and questionnaire results. Students generally found using the app a positive experience, which made lectures and tutorials more interactive and fun.

  • They did not like using Android devices if they normally used iOS, even if loaned one.
  • The integration of voting and wagering into the app was very popular.
  • Mobile phone problems and a lack of understanding of the app held some students back so they felt they then couldn’t catch up with the rankings of their peers.
  • They appreciated the flexibility of the app, which meant they could use the parts they liked and ignore those they didn’t like.
  • Some students missed instruction sessions and explanatory emails and then didn’t understand what the app was for and didn’t ask.
  • Some compared the app with commercial games and had extremely high expectations of quality and stability that weren’t met.
  • Some were embarrassed to admit they like video games and played video games, which had an impact on how they interacted with Unicraft2 and staff.
  • Some were familiar with quite complex video games and wanted the same level of depth.
  • Some suspected others of cheating and just the suspicion of it was enough to annoy them.
  • Many students felt it helped their engagement, made lessons more fun and increased their awareness of the progress they were making.

 

 

Conclusion

Unicraft2’s constructive competition based video game mechanics, caused engagement with the app to increase significantly and reduced the rate at which student’s lost interest (see Figure 8). There is no evidence that this gamification project negatively affected motivation (see Table 2). There is evidence that Unicraft2 increased attendance for the FOP subject compared to other subjects and by 11% compared to the same subject in the previous year (see Figure 7). There is evidence that interactive quizzes increase attendance and that Unicraft2 maximises that increase (see Figure 7). There is evidence that Unicraft2 caused an increase in student attainment of 7% (see Figure 10). These results support the study’s three hypotheses: when gamification projects use more game mechanics, look more like videogames and use only constructive extrinsic motivators, participants are more likely to engage and stay engaged without loss of intrinsic motivation.

References

  • Artino, A.R.. J., 2005. Review of the Motivated Strategies for Learning Questionnaire. Online Submiss.
  • Behm-Morawitz, E., 2013. Mirrored selves: The influence of self-presence in a virtual world on health, appearance, and well-being. Comput. Human Behav. 29, 119-128.
  • Deci, E.L., Ryan, R.M., Koestner, R., 1999. A Meta-Analytic Review of Experiments Examining the Effects of Extrinsic Rewards on Intrinsic Motivation. Psychol. Bull. 125, 627-6.
  • Deterding, S., Sicart, M., Nacke, L., O’Hara, K., Dixon, D., 2011. Gamification. using game-design elements in non-gaming contexts, in: CHI’11 Extended Abstracts on Human Factors in Computing Systems. ACM, pp. 2425-2428.
  • Featherstone, M., 2017. A formative evaluation of a gamification app containing asynchronous multiplayer game elements, in: Proceedings of the 11th European Conference on Games Based Learning, ECGBL 2017.
  • Fuchs, M., Fizek, S., Ruffino, P., Schrape, N., 2014. Rethinking gamification. Meson Press.
  • Fulop, M., 2009. Psihologijske teme., in: Psihologijske Teme. Odsjek za psihologiju, Filozofski fakultet u Rijeci, pp. 345-367.
  • Raftopoulos, M., Walz, S., Greuter, S., 2015. How enterprises play: Towards a taxonomy for enterprise gamification. Conf. Divers. Play.
  • Turning technologies, 2015. Turning Technologies: Interactive Learning; Audience Response Technology.

 

 

 

 

 

Back to list of abstracts