Clavis Aurea ” : An augmented Reality Game for the Teaching of Local History

DOI: http://dx.doi.org/10.24018/ejers.2020.0.CIE.2310 December 2020 1 Abstract—The aim of this study was the design, development, and evaluation of an Augmented Reality game to teach students about the local history of a Greek island. Design-based research served as the foundation of this study’s methodology. Experts in ICT in education, teachers with knowledge of the local history and primary education students participated in the evaluation of this study. The results of the evaluation showed that the game presents satisfactory levels of usability and that its content is compatible with the island’s local history. Among the factors influencing its use by students were internet connectivity, the large number of visitors at the archaeological site where the game was played, and the problem of coordinating the student groups.


I. INTRODUCTION
The subject of History occupies an essential place in the curricula of all education systems. According to [1], the aim of teaching this subject is to prepare young people for their upcoming participation in a pluralistic democracy. Nevertheless, in practice this subject is viewed as "non appealing" and related to many non "useful" details [2]. Factors contributing to this are, among others: a) the teachercentered approach that frequently compels students to memorize events [3], b) the theoretical fabric of curricula that "overlooks" the fact that there are qualitative differences between the thinking of children and adults [4], and c) the adoption of "static" approaches which hold that History should not be subject to interpretations, but rather, that it represents an accepted understanding of the past [5].
One of the proposals included in curricula regarding the teaching of history -local history in particular -involves educational trips to archaeological sites. It is argued that these facilitate the placement of students in a more clear-cut cultural context, as well as the fact that they contribute to students' functional engagement with the subject of History [6]. Furthermore, these sites represent authentic learning environments, whose historical value cannot be articulated merely by speaking about them in the classroom [7]. Although these trips are frequently associated with expectations, nevertheless, the actual experiences of the students end up being "non appealing" [8], [9], inasmuch as the students are given a tour of "ruins and relics" from a past era that hardly resemble their original form [10].
Augmented Reality (AR) [11], [12], [13], [14], [15] could serve as a solution to this problem. It "supplements the user's sensory perception of the real world by the addition of computer-generated content to the user's environment and offers a new form of interactivity between real and virtual worlds" (p. 2) [15]. Specifically, AR games [16] using the placement and viewing of digital objects in the real world are ideal for teaching and learning in areas with rich local history and culture [17], [18]. AR games are played in the real world with the support of digital devices (e.g. smart phones, tablets, smart glasses) that create a fictional layer on top of the real world context [19], [16]. The showcasing, reenactment, observation, and discovery of an area's historical and cultural content through AR games can prove conducive to situated learning and the strengthening of an informal learning environment's value in the teaching of local history. Despite the research interest observed regarding the design and development of AR mobile games related to the teaching of local history, the literature on the use of such games in primary education is nonetheless limited.
The aim of this study was the design, development, and evaluation of an AR mobile game, "Clavis Aurea", for the teaching of local history. The archaeological site of the Castle of Naxos, a Greek island, served as the case study. The objectives of this study were: a) the formation of a framework for the design and development of the game "Clavis Aurea", b) the evaluation of the game's usability and content, and c) the investigation of the factors affecting the game's use in an archaeological site.
This article begins by presenting examples of AR games for the teaching of local history (Section 2). Then, it presents the methodology of the design, development and evaluation of the game "Clavis Aurea" (Section 3). Section 4 describes the main features of the game. Section 5 presents the results and data analysis of the evaluation of the game. Finally, conclusions are outlined in Section 6.

II. AR GAMES FOR LOCAL HISTORY
Nowadays, there is a growing number of studies about teaching local history on mobile technology devices that refer to either the development of AR games [18], [20], [21], their formative evaluation [22], [23], [24] or their effects on learning [25]. These games take place in various sites that possess historical and cultural value. For example, [26] developed the treasure hunt game "O'Munaciedd," whose aim was to obtain knowledge about the historical and cultural heritage of the city of Mantera. In the study by [27], the hidden treasure game "Oracle of Delphi app" was presented, and it was played in the archaeological site of Delphi, Greece. Galatis et al. [10] designed the AR game "KnossosAR," which takes places in the city of Knossos on the island of Crete, Greece, and aims to bring students in contact with the Minoan Civilization. In a study by [28], the game "SejarAR" was used to teach about Malaysia's local history. In a more recent study, [29] developed the AR game "HistoriAR", which aims to teach users about the local history of Indonesia.
The studies focusing on AR games' effects on learning showed positive outcomes [30], [25]. For example, one of the first games designed for outdoor use was "Reliving the Revolution" [31]. The aim of this game, which targets secondary education students, is to enable them to "relive" the events of 1775 at Lexington, Massachusetts, which was the site of a battle during the American Revolution, and to decide who "fired the first shot" in the battle. With the aid of PDA devices, players in groups take on different roles and interact with historic figures and objects that are activated on the screen via GPS, depending on their location. Three different groups participated in the evaluation of this game. The first two consisted of university students and graduates, while the third consisted of local middle and high school students. The results showed that AR games are capable of engaging students to genuinely practice critical 21st century skills. In another study, [25] studied the effects of the AR game "Frequency 1550" for smart phones, tracking the engagement of 458 students (ages 12-16) with it, their knowledge about history and their motivation regarding local history in general; in particular, about Medieval Amsterdam. The results of their study showed that students who played the game were more engaged and had greater knowledge on Medieval Amsterdam compared to the control group.
In a more recent study, [30] examined the ability of the AR game "Jewish Time Jump", which was developed on the AR platform ARIS for mobile devices, to foster connections between historic and current events that relate to marginalized groups within the general population (e.g., immigrants). This game takes place in a park in New York. The game targets students ages 10-14, who, with the aid of their device, take on the role of reporters and travel back in time to the year 1909 in order to cover the story of a strike that was largely spearheaded by young female workers of Jewish descent. As part of this effort, the students interview the protagonists of the events who appear on their device, while at the same time, the storyline of the game adjusts accordingly, based on their selections. The evaluation took place over three recurring rounds on a sample of 45 students. The game had a positive effect in sensitizing participants to current events, such as poor labor conditions for specific social groups around the world.
In addition, a literature review of studies on AR games showed that the majority of them focus more on secondary school students as sample groups [25]. Studies on AR games for primary education are limited [23]. This study aims to fill this research gap.

EVALUATION OF THE GAME
The design, development, and evaluation of the game "Clavis Aurea" followed the methodology of Design-Based Research (DBR). A similar methodology has been used in previous studies related to AR in education [30], [32], [33], [34]. The stages used in this study are presented in Table 1 and described in the following sections. Development of the game using the "ZAPworks" and "Actionbound" platforms.
Formative evaluation of the game • Evaluation of the game's usability, pedagogical design and content compatibility with the elements of local history by ICT in education experts, teachers and students.
Evaluation of the game in the real-world setting • Examination of the factors influencing its use in real-world situations, as well as students' views regarding this experience.

A. Recognition and analysis of the problem
This stage focused on analyzing the need for creating an AR game for local history. This included: a) a review of the literature in the English language (Science Direct, Scopus and ERIC) regarding the use of AR at archaeological and historical sites, b) the study of the Greek curriculum and school textbooks regarding the instructional framework for local history and c) the specification of the game's aim and objectives. The archaeological site of the Castle on the island of Naxos, Greece served as the case study for the development of the game. This site has various museums, and primarily features cultural influences from various civilizations (e.g. Cycladic, Byzantine). Moreover, the archaeological site offers the opportunity for independent touring and exploration across multiple trails.

B. Design and development of the game
In this stage, the game scenario based on the features of the archaeological site of the Castle of Naxos, the museums that are situated there, and the individuals who played an important role in the area's local history was initially formulated. Three teachers from the area, with knowledge of the culture and local history of Naxos, also helped to formulate the content of the scenario. At the same time, the pedagogical and design principles of the game were also chosen. Elements of the theory of social constructionism, as well as design principles for AR games were used [35]. These were the principles of curiosity, challenge, imagination, as well as the design principles for multimedia learning [36], such as the "segmenting principle", the "coherence principle", the "multimedia principle", the "space contiguity principle" and the "temporal contiguity principle".
Finally, the game was developed using the "ZAPworks" and "Actionbound" platforms. These provided the user with the ability to interact with both visual and audio objects, as well as corresponding image and location-based AR objects. More specifically, "Actionbound" was used to develop the game for the purpose of guiding students via GPS to the areas of the Castle that had historical and cultural value, and to augment them with images, videos, texts, audio, questions, and quizzes. In addition, the students could use the application to gather audiovisual materials (e.g., interviews, videos, photos) from the areas they visited, depending on the purpose of their mission at every stop. Using ZAPworks, various cards, brochures, and posters were augmented with images, audio and videos. These materials were placed in various areas of the Castle and students were asked to discover them and scan their contents, which revealed a variety of details relating to the continuation of the game.

C. Formative evaluation of the game
In this stage, the first edition of the game was evaluated for usability, pedagogical design, and content compatibility with the elements of local history. 28 experts in ICT in education (i.e., in-service teachers with postgraduate studies in ICT in education) participated in the evaluation of the game's usability. This was a convenience sample that was chosen based on members' scheduling availability to participate in the study. Among them, 21 (75%) were females and 7 (25%) were males. 23 (82.1%) worked in primary education and 5 (17.9%) in secondary education. Their mean age was 38.4 (SD=7.545).
After interacting with the game on their smart phones, they participated in discussions and interviews regarding its usability based on the following heuristics rules [37]: visibility of system status (e.g., Does the game display always begin with a title or header that describes screen contents?), aesthetic and minimalist design (e.g., Are all icons in a set visually and conceptually distinct?), match between system and the real world (e.g., Are icons concrete and familiar?), consistency and standards (e.g., Is the most important information placed at the beginning of the screen?), error prevention (e.g., Are menu choices logical, distinctive, and mutually exclusive?), and flexibility and efficiency of use (e.g., Does the game offer "next" and "previous" buttons?). In regard to the content and its compatibility with the local history, the game was evaluated using interviews with three teachers (two women and one man, ages 45-50) who worked in local schools and were knowledgeable on the cultural heritage of Naxos and the features of the archaeological site at the Castle. Finally, the game was evaluated for ease of use by five students from a school in Naxos. Data were also gathered through observation of the experts, teachers, and students as they interacted with the game.

D. Evaluation of the game in the real-world setting
In this stage, the game was played at the archaeological site of the Castle by 26 fifth-graders from a primary school in Naxos for the purpose of examining the factors that influence its use in real-world situations, as well as students' views regarding this experience (see Fig. 1). boys. This student sample was chosen because their school was located very close to the Castle, which served as the site of the game. It was easier and safer to transport them to the site compared to any other school on the island of Naxos. The students were accompanied by their two classroom teachers. The duration of the game was 2 hours. Data were collected from worksheets that the students completed after the game and by observing them while they played the game. More specifically, among other things, the worksheets asked students to write about how they felt playing the game, problems they encountered during their missions at the Castle, and if they would like to play the game again.
IV. THE GAME "CLAVIS AUREA" The game "Clavis Aurea" (which means golden key in Latin) deals with the local history of Naxos, an island in Greece, and takes place in the archaeological site of the Castle. It is designed for students ages 9-12 and is played in groups of 2-3 persons. In order to learn about local history, students must search for the golden key that will transport them to the island's past. Using smart phones or tablets, they gather real and virtual information and facts about the Castle, and they work together to solve riddles and problems. More specifically, the game began inside the classroom. There, the students were presented with a glass bottle that was discovered within the framework of an environmental campaign to promote recycling. Hidden inside the bottle was an old clipping displaying a map of the island of Naxos and the medieval castle that served as the duchy of the archipelago during the Venetian period. Based on the first clue, the students were asked to go to the northern gate of the Castle (Great Gate of the Venetian Castle). There, they were asked by pirate Markos Sanoudos to find the treasure that he himself had hidden in the Castle before he died. As they looked for the treasure, the students had to solve a series of problems with the help of a parrot named "Captain Axos". To achieve the aim of the game, students must visit sites of major cultural interest (e.g., the Venetian Folk Museum, the Byzantine Museum, the Archaeological Museum) using the application "Actionbound" in order to come into contact with the Venetian period, Byzantine culture, and Cycladic art (see Fig.  2). If the students successfully manage to find all the clues, they will end up at the southern gate of the Castle (Paraporti Gate), where Markos Sanoudos' treasure is hidden, and where the students will see him appear on the screen of their mobile device with the help of AR technology. The augmented objects, which are placed using QR codes in various areas of the Castle, appear to the students through the use of operations that are provided by the platforms used to augment the game, "ZapWorks Designer" (https://zap.works/designer/) and "Actionbound" (https://en.actionbound.com/).

V. RESULTS
According to the experts in ICT in education, the game was easy to use and usable. The outcomes of its usability, in terms of minimalism, showed that it did not contain excessive information, functions, or complex graphics. Some representative answers were the following: Users can learn to play the game very easily, since the directions given throughout the game are not too complicated. (ICT expert 2) …it is not necessary for players to possess prerequisite knowledge in order to run the existing application. (ICT expert 18) The game attempts to avoid the cognitive overload of players by combining various types of multimedia. (ICT expert 22) The environment of the game has a good degree of userfriendliness, as it does not overload players with excessive information in order to disorientate them. (ICT expert 13) The language was evaluated as being understandable and appropriate for the students' age range. The application's demand on users' memory load and its consistency of flow on the screen were also positively evaluated. The following answers were selected as typical examples of ICT experts' views: …the vocabulary used in the application is suitable for the age of the players whom it targets. The directions given during the game are understandable and clear. (ICT expert 25) Activities are introduced in the game seamlessly and help players adjust accordingly based on how the storyline develops. (ICT expert 21) …the visual and verbal code is frequently combined on the monitor throughout the game. (ICT expert 11) The players do not have to memorize the various messages that will help them find the treasure for a long time. Usually, stages in the game are completed right after the acquisition of new information related to the search for the next area of historical interest. (ICT expert 1) The provision of interactivity was further enhanced through the incorporation of more audio signals as users approached areas of interest or through the augmenting of content following the scanning of QR-codes.
…the use of augmented reality technology can significantly raise players' curiosity. (ICT expert 15) It is apparent that players' contact with real and virtual elements combined with the relief of a medieval castle increases their desire to engage more closely with the game. Based on the observation of students and discussions with them, it appeared that they found the application very easy to use. Some of the students' statements from the discussion are as follows: From the start of the game, my team was able to use the application with ease, as the steps were simple and clear. (Student 3) …it was not necessary for us to request help from any of the teachers accompanying us to the Castle. It was very easy for us to use the mobile devices that they gave us. (Student 9) …I think that younger students could also play this game since all of us now play many games on our mobile phones. (Student 14) Also, the recommendations made by the teachers and experts in ICT in education during the group discussion regarding the pedagogical design had to do with the enhancement of the game with more collaborative activities involving visitors to the Castle, as well as museum employees located on site. Two representative responses given by them are as follows:

It would be a very good idea to have the users come into contact with the tourists visiting the Castle so that they can develop collaboration and communication skills. (ICT expert 1)
Adults with knowledge about the history of the Castle of Naxos, such as the employees at the Byzantine and Archaeological Museum, could play an active role. I believe that by engaging in collaborative activities with them, the players could gain significant knowledge and experiences regarding their local history. (Teacher 2) The three teachers with knowledge in local history reported that all its areas were addressed by the game, and that its content was suitable for the students. For example, among other things, they mentioned the following: There The path that students are asked to follow consistently connects all the museums and the areas that a visitor should look for in the Castle. (Teacher 3) Moreover, the three teachers noted that its playful storyline can lead students to learn about the island's local history in a more empirical and pleasant way. For example, the following opinions were expressed: This is a game that heightens the sense of mystery and students' curiosity. The idea of searching for a hidden treasure in their local area motivates them to interact with the game to a great degree. …Voices of pirates, narrow passages, elements that suddenly come to life through the screen of a mobile device. This is a playful approach to learning about our local history. (Teacher 3) The game "Clavis Aurea" gives students the ability to "escape" from the boundaries of the formal classroom setting and actively engage in a game by assigning roles. …The fact alone that they were able to use mobile devices piqued their interest to participate in the game more actively.

(Teacher 1)
In terms of students' views on the game, the results showed that they felt enjoyment, enthusiasm, and satisfaction, all of which contributed to increasing their motivation to engage in the game more actively. Furthermore, the students' attitudes regarding the use of AR technology were positive, and the majority of them stated that they were impressed by their interaction with it. Some of their statements are presented below.
I felt so happy when I found the first clue to the riddle and held the envelope with the secret codes in my hands. (Student 6) This was the first time that I was asked to play a game using mobile devices in school. I was excited to see objects that were not shown on paper come to life on the screen. (Student 10) …The first thought that came to mind was that I would want my team to be the first to find the hidden treasure in my island's castle. (Student 12) I wish that this game could have lasted longer and that we could have had even more hidden clues. (Student 17) This was a unique experience that I would like to try again in the future with more difficult missions. (Student 23) Augmented reality is amazing. I did not know that such a capability existed. I was impressed by the fact that we could be walking in the castle and at certain points, we would suddenly hear audio messages and were able to scan various QR codes and pamphlets with our mobile phone, and these would come to life with images, text, video, and websites. (Student 24) Moreover, it was ascertained that the features of the archaeological site of the Castle and the fictional characters (e.g., pirates) that were used enhanced the game's appeal even further, increasing students' interest. Some of the students' statements are as follows: We were walking along the narrow passages of the Castle and would suddenly hear the voice of the pirate Markos Sanoudos from our device. I felt like he was following us.... (Student 14) …I would be so proud if I were the first one to find the treasure. I too could become a successor to Markos Sanoudos! (Student 6) …it was so lovely to find various messages located on the secret riddles inside our Castle. It was as if they had been left by real pirates. ( Based on the data that arise from the observation of the students' participation and the worksheets, several factors that affected the flow of the game were found. Among these were the problem of maintaining internet connectivity at the archaeological site where the game took place. At one point, I thought of quitting the game, since it was not easy to read the secret messages at some points. The internet connection was interrupted and we called over our teacher to help us. (Student 22) …the only area where I would like the game to be improved is the internet. At the same time, difficulties in accessing all the areas housed in the Castle of Naxos were reported due to the large number of visitors that were present at the time when the students were invited to interact with the game.
…there were so many tourists at the Castle that at one point we lost the other team that was nearby us. Fortunately, our teacher helped us. (Student 2) I would have preferred that we play the game at another time, since we were unable to run through the narrow passages. Visitors were walking around and taking pictures of the monuments. (Student 5) At one point, a visitor complained to us because we were yelling out of happiness over finding a hidden clue. (Student 12) …due to the number of visitors at the Byzantine Museum, we were not allowed into all the galleries. (Student 16) At certain points, there were large crowds gathered at the Castle and as a result, we were not able to hear the audio messages on our mobile device. (Student 17) Finally, the two teachers accompanying the students to the Castle noted the problem of establishing good coordination with the student groups, which, together with the two aforementioned factors, created problems with the flow of the game. In particular, the teachers reported coordinating difficulties due to the large number of student groups playing the game at the same time.
I think that it is very difficult for us as teachers to coordinate 26 students at a time (the month of September) when tourist traffic in our island is still high. We were very anxious over the possibility of a student getting lost in the crowd. Moreover, I want to emphasize the fact that due to the noise the students were unable to hear the audio messages and concentrate on the activities of the game. I believe that these types of games should be played in the winter or early spring when we don't have tourists. (Accompanying teacher 1) I think that the game should be played with the help of parents as well. They could play the role of chaperone and accompany two student teams at a time. This way, we teachers could help with questions regarding the content and storyline of the game instead of worrying about the safety of the students. Moreover, to solve the issue with the internet, we could have used the museums' Wi-Fi connection. We hadn't foreseen this prior to the start of the game. (Accompanying Teacher 2)

VI. CONCLUSION
This study dealt with the design, development, and formative evaluation of the AR mobile game "Clavis Aurea" for the local history of Naxos. The results showed that the game was evaluated as easy to use and usable by both experts in ICT in education, as well as by teachers and students. The game's usability is likely due to the fact that it was enhanced with audiovisual materials, as well as messages from fictional figures, such as the pirate Markos Sanoudos. According to previous studies, these elements increase the degree of digital games' usability and improve their flow [38], [39]. Moreover, one of the conclusions arising from this study is that the features of the archaeological site of the Castle, in which the AR game takes place, may have affected students' motivation to explore the area and complete all the missions contained in the game. Similar results were found in previous studies [40] that used local medieval castles as the primary scene of action for the players. The results from the interviews with teachers showed that the game "Clavis Aurea" is compatible with the area's local history. Similar conclusions were arrived at from the findings of studies [41], [42] that used AR games in teaching to showcase local cultural heritage.
Furthermore, based on the results from the evaluation of the game, it was discovered that various factors (i.e., problems with internet connectivity, the large number of visitors, the difficulties coordinating so many student groups) can obstruct or suspend the normal flow of the game. Similar technical or organizational problems were found in previous studies [41]. Based on this, the conclusion could be drawn that the provision of support to students from teachers during their interaction with the game can be considered important. This is consistent with a study by [43], in which the need to guide students during the evaluation of digital games is highlighted. Similarly, in studies by [44] and [45], it is mentioned that during the learning process AR games should operate under the supervision of a teacher so that students may be taught and not distracted.
Another conclusion that arises from this study is that location-based AR games, such as "Clavis Aurea", can serve as a useful instructional tool for the teaching of local history. Corresponding conclusions were formed based on the findings of studies [41], [42] that used mobile AR games in the learning process to showcase a locality's cultural heritage. The students were in a position to understand the historic events of their local area more easily and familiarize themselves with their culture using a more experiential and entertaining medium as compared to traditional teaching methods. Furthermore, in the English-language literature, researchers have focused on the design of AR mobile games for local history [46], [27], [20], [47], [40] due to the added value that they provide to the learning process. Students are able to engage in more complex activities and gain new knowledge in an exploratory framework. This study enriches the existing bibliography for both location-based AR games that are played using mobile devices, as well as games examining local history for primary education students. Nonetheless, this study has two limitations First, the sample that participated in all the stages of research was convenience based. Second, due to constraints on time and resources, the study did not collect data regarding the effect of the game on students' knowledge of local history. Future studies must take these limitations into account. Moreover, future studies could examine the acceptance of such games by teachers and the factors that affect their intent to use them. It would be of particular interest to examine what sort of interactions take place among the student teams and what sort of activities are needed within the context of constructivism and situated learning to make learning and the experience itself more interactive and discoverable. Finally, these types of games which are played in cultural heritage sites could utilize smart glasses, in addition to mobile devices, to examine the degree of students' immersion during the use of augmented reality.