Towards modelling of Mixed Reality application for language learning for healthcare – part 2

Related work

Defining Mixed Reality (MR) as a term has not seen a consistent use. In some contexts it includes both AR and VR, whilst in others, it represents the point where real and virtual elements are equally displayed for the user (Speicher et al., 2019).  According to Speicher et al. (2019), MR combines different dimensions: (a) number of environments (e.g. Virtual Reality, Augmented Reality), (b) number of users (e.g. one or many), (c) level of immersion (e.g. non immersive, partly immersive, fully immersive), (d) level of virtuality (e.g. non virtual, partly virtual, fully virtual) and (e) degree of interaction (e.g. implicit and explicit). For the purposes of this work MR is classified as combination according to Speicher et al. (2019), where distinct AR and VR parts are combined and interact with each other, but are not tightly integrated and user can switch between Augmented Reality (AR) and Virtual Reality (VR) devices as necessary.

The benefits of Mixed Reality in education

Applications for Mixed Reality (MR) have seen a rise in popularity in recent years in a variety of fields including education and medicine (Barba & Marroquin, 2017). Extended realities such as MR, VR and AR can offer learning methods that would not be available otherwise, allowing for a more hands-on, immersive, and active learning approach to teaching and learning (Allcoat et al., 2021). We chose MR over other digital tools or media such as web or video content due to its affordance to provide in-situ interactions for enabling real-like communication in healthcare contexts. MR environments have the ability to promote learning by giving students the advantages of engaging users with interactive content (Yannier et al., 2015). By providing intuitive interactions with data in our living spaces and with our friends, MR frees us from experiences that are limited to screens (Bray & Zeller, 2018). The combination of the physical and digital worlds, enable fluid and natural 3D interactions between people, machines, and the environment.  Beyond displays, mixed reality applications now also include spatial mapping and anchors for environmental understanding, hand, eye, and speech input tracking, sound in space, both physical and virtual spaces’ positions and locations as well as collaboration on 3D assets (Bray & Zeller, 2018). MR is a suitable and safe learning technology that has the potential to enable new approaches to teaching. The advantages, such as increased engagement and positive emotions, indicate that MR would be beneficial as supplement to traditional learning methods (Allcoat et al., 2021).

Past studies have also demonstrated enhanced learning rates and skills improvement as a result of the induced sense of immersion and presence in such a ‘risk-free’ mixed environments. In addition, extra cues in the form of visual or auditory feedback can facilitate learning of the task and allow simulating the task in a flexible way to adapt it to users’ needs and training goals (Carver & Lamb, 2020; Zheng et al., 2022). Considering these advantages of MR, our work is focused on designing an educational application for the healthcare professionals with a clear orientation toward the final outcomes of the learning experience.

Mixed Reality and Language Learning

Mixed reality has the potential to improve language learning by offering a more immersive, fun, and tailored learning experience that can aid students acquire new language skills faster (Zhang et al., 2023; Hsu et al., 2023). MR-enhanced language learning can be implemented under different theoretical frameworks such as game-based learning theory, cognitive theories of multimedia learning, sociocultural theory, collaborative learning theory, constructivist theory, and embodied learning theory  (Zhang et al., 2023).  Incorporating Mixed Reality (MR) into language learning entails multiple advantages such as increased autonomy, improved communication and collaboration, reduced risk for learners, rich input, opportunities for practice, support communication and collaboration, improved student-centeredness, authentic language use and increased enjoyment (Chen & Tsai, 2009; Zhang et al., 2023; Hsu et al., 2023).

Popular VR, AR and MR simulators for language learning (e.g. “Do you speak Holo?” in Torelli et al., 2020; Virtual Medicine, 2023; EduVenture VR – Apps on Google Play, 2023; Second Life – Virtual Worlds; Virtual Reality, VR, Avatars, and Free 3D Chat, 2023; Google Expeditions, n.d.; OpenSimulator, 2022; Mondly AR, 2019 etc.) have gained popularity in recent decades and have been integrated into self-paced, blended and face to face language courses (Lin et al., 2015; Parmaxi, 2020; Torelli et al., 2020; Statti & Torres, 2020; Zhang & Zou, 2022). Additional simulators for language learning for specific purposes in the context of healthcare (e.g. Ein Tag Deutsch in Der Pflege, 2017) provide real-life practice for learners to improve the necessary cognitive, affective, and psychomotor skills, however the majority of them have limited content focusing mostly on virtual simulations of healtchare equipment and protocol training rather than language learning and/or communication. Moreover, there are various simulators available for healthcare professionals but do not aim to support language learning (e.g. Sharecare VR, 2023; Osso VR, 2023; Oxford Medical Simulation, 2023). For example, Oxford Medical Simulation (Oxford Medical Simulation, 2023) is a VR nursing simulation which despite promoting communication between healthcare professionals, it does not aim at supporting language learning.


Development of the DRFLEMP MR application for Healthcare

The involvement of dedicated stakeholders is considered important towards the design and development of an application that is expected to support the language learning experience of professionals in the healthcare sector (Howard et al., 2021). With this in mind, we collaborated with multimedia specialists, instructional designers, healthcare instructors, language instructors and students over a period of 10 months to gather a deep understanding of the needs and the perspectives of the perspective users. By incorporating the ADDIE model, we emphasized on a systematic and iterative process for creating effective learning experiences. The ADDIE acronym represents the five key stages of the model, that is Analysis, Design, Development, Implementation, and Evaluation. We commenced the MR application with the Analysis phase. This involved working collaboratively with multimedia specialists, instructional designers, healthcare instructors, language instructors and students to identify learning needs, goals, and objectives for the DRFLEMP MR application. Moving on to the Design phase, we implemented Design Thinking workshops with our key stakeholders and developed specific learning objectives aligned with the identified learning outcomes. Additionally, we crafted evaluation strategies to assess the learners’ skills effectively. With the learning objectives and evaluation strategies in place, we progressed to the Development phase. During this stage, again through Design Thinking workshops we designed the instruction in the form of Modularised Units. These modules were carefully structured to include guided activities, practice session, enabling learners to practice language in a meaningful and organised context. Finally, the mudularised units that derived from the Design Thinking Workshops were Evaluated and feedback received from learners and instructors informed revisions and refinements of the MR application. The resulting learning activities effectively fulfilled the derived learning outcomes, as depicted in Figure 2. Throughout the process, we followed an iterative approach, continuously refining and improving the system design and development of the modularized units for the DR FLEMP Healthcare application to ensure an optimal language learning experience.

Table 1. Development of the DRFLEMP MR application for Healthcare

 Analysis: Market Research, Survey and Discussions with Key Stakeholders

In the analysis phase, AR, VR and MR platforms were analyzed through systematic literature review and survey with consideration given to technological features, sustainability as well as technological and pedagogical opportunities. With the increased use of AR, VR and MR applications, various applications are currently available. This provided the team with concrete examples of the existing MR applications, as well as their strengths and weaknesses.  These technologies have been observed to enhance students’ creative thinking efficacy (Y. J. Lin & Wang, 2021) and facilitate the advancement of language proficiency and communication skills among students (Jehma, 2020). The significance of authentic contexts and lifelike settings was also highlighted (Ma, 2021; Hara et al., 2021; Taguchi, 2022). These environments provide valuable opportunities for interactive learning, which in turn promote enriched learning experiences and improved educational outcomes (X. Chen et al., 2022). The desk research analysis also highlighted the provision of instant feedback (e.g. Mondly VR, 2021; Mondly AR, 2019), the availability of content into various levels (e.g. ImmerseMe,, 2023), the accessibility from anywhere and compatibility with various devices such as mobile, desktop, VR headset (e.g. Bodyswaps, 2023), the interaction with other people online (e.g. FluentWorlds, 2023), the gamification elements (e.g. Busuu,2018), as well as some or entire free functions of the applications (e.g. Ein Tag Deutsch in Der Pflege, 2017). Some other advantages identified, include the use of authentic scenarios (Hamad & Jia, 2022) (e.g.ImmerseMe, 2023; Ein Tag Deutsch in Der Pflege, 2017), the development of communication skills (e.g. VirtualSpeech, 2021), the opportunity to practice in multiple languages (e.g. ImmerseMe, 2023;  Dynamic Languages, 2023), as well as the customizable and flexible environment (e.g. Avakin Life, 2023; UbiSim, 2023) and the provision of immersive features such as 3D models and 360 videos (e.g. 3D ORGANON, 2023; ImmerseMe, 2023; Dynamic Languages, 2023). However, it’s important to note that while there are benefits, there are also potential challenges and pitfalls associated with the application of XR technologies. Notably, common barriers encountered include technical difficulties, motion sickness (Jehma, 2020; Wu et al., 2021; Kruk, 2021) limitations in time and space  (V. Lin et al., 2022), diminished confidence and motivation levels  (Kruk, 2021), as well as heightened anxiety when engaging with XR applications (J. C. C. Chen, 2020). Adittionally, the desk research identified the paid functions or applications (Alfarsi & Yusof, 2020) (e.g. Dynamic Languages, 2023; UbiSim, 2023), the limited content availability (Hamad & Jia 2022) and only in specific languages such as English (e.g. VirtualSpeech, 2021;FluentWorlds, 2023) or specific language aspects such as vocabulary (e.g. Panolingo, 2017), limited CEFR levels offered (e.g. Ein Tag Deutsch in Der Pflege, 2017; Varvara, 2017), lack of updates (e.g. Panolingo, 2017) and availability only in specific countries or specific population (e.g. Varvara, 2017). Furthermore, some applications were found to behard to operate and maintain (Hamad & Jia, 2022)(e.g.UbiSim, 2023), used non-realistic characters (e.g. Osso VR, 2023) and require specific equipment such as standalone headsets (e.g. Virtual Medicine, 2023).

Discussions within the key stakeholders including 5 multimedia experts, 1 healthcare instructor and 8 language instructors raised awareness of the needs and challenges of the DR FLEMP application, as well as its vision. At this stage, we also developed user personas to represent exemplary users of the DR FLEMP MR application, allowing the design team to develop a deeper understanding of the target users, their needs, preferences, and behaviors. Moreover, the discussions with key stakeholders brought forward the features and functionalities they considered important for the DR FLEMP MR application which were grouped according to common themes and were prioritisd following the MoSCoW prioritisation technique in order to  reach a hierarchy of features. This phase concluded with a survey to healthcare instructors, healthcare professionals, language instructors and healthcare students to inform the design team regarding the learning needs, goals, and objectives of the DRFLEMP MR application. Having concluded this phase, the collected recommendations and needs we reached a consensus on the vision of the DR FLEMP MR application.