collaborative problem solving in local and remote vr situations

Collaborative Problem Solving in Local and Remote VR Situations

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Collaborative Problem Solving in Local and Remote VR situations

Mps-authors.

Hatzipanayioti,  A Department Human Perception, Cognition and Action, Max Planck Institute for Biological Cybernetics, Max Planck Society; Max Planck Institute for Biological Cybernetics, Max Planck Society;

Pavlidou,  A Department Human Perception, Cognition and Action, Max Planck Institute for Biological Cybernetics, Max Planck Society; Max Planck Institute for Biological Cybernetics, Max Planck Society;

Bülthoff,  HH Department Human Perception, Cognition and Action, Max Planck Institute for Biological Cybernetics, Max Planck Society; Max Planck Institute for Biological Cybernetics, Max Planck Society;

Meilinger,  T Department Human Perception, Cognition and Action, Max Planck Institute for Biological Cybernetics, Max Planck Society; Max Planck Institute for Biological Cybernetics, Max Planck Society;

External Resource

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8798201 (Publisher version)

Fulltext (restricted access)

Fulltext (public), supplementary material (public).

Hatzipanayioti, A., Pavlidou, A., Dixken, M., Bülthoff, H., Meilinger, T., Bues, M., et al. (2019). Collaborative Problem Solving in Local and Remote VR situations. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (pp. 964-965). Piscataway, NJ, USA: IEEE. doi:10.1109/VR.2019.8798201.

Collaborative Problem Solving in Local and Remote VR Situations

• Open supplemental data
• Reference Manager
• Simple TEXT file

People also looked at

Review article, virtual reality and collaborative learning: a systematic literature review.

• 1 Centre for Education and Learning, Delft University of Technology, Delft, Netherlands
• 2 Leiden Institute of Advanced Computer Science, Leiden University, Leiden, Netherlands
• 3 Interactive Intelligence, Delft University of Technology, Delft, Netherlands

Background: While research on Virtual Reality’s potential for education continues to advance, research on its support for Collaborative Learning is small in scope. With remote collaboration and distance learning becoming increasingly relevant for education (especially since the COVID-19 pandemic), an understanding of Virtual Reality’s potential for Collaborative Learning is of importance. To establish how this immersive technology can support and enhance collaboration between learners, this systematic literature review analyses scientific research on Virtual Reality for Collaborative Learning with the intention to identify 1) skills and competences trained, 2) domains and disciplines addressed, 3) systems used and 4) empirical knowledge established.

Method: Two scientific databases—Scopus and Web of Science—were used for this review. Following the PRISMA method, a total of 139 articles were analyzed. Reliability of this selection process was assessed using five additional coders. A taxonomy was used to classify these articles. Another coder was used to assess the reliability of the primary coder before this taxonomy was applied to the selected articles

Results: Based on the literature reviewed, skills and competences developed are divided into five categories. Educational fields and domains seem interested in Virtual Reality for Collaborative Learning because of a need for innovation, communities and remote socialization and collaboration between learners. Systems primarily use monitor-based Virtual Reality and mouse-and-keyboard controls. A general optimism is visible regarding the use of Virtual Reality to support and enhance Collaborative Learning

Conclusion: Five distinct affordances of Virtual Reality for Collaborative Learning are identified: it 1) is an efficient tool to engage and motivate learners, 2) supports distance learning and remote collaboration, 3) provides multi- and interdisciplinary spaces for both learning and collaborating, 4) helps develop social skills and 5) suits Collaborative Learning-related paradigms and approaches. Overall, the reviewed literature suggests Virtual Reality to be an effective tool for the support and enhancement of Collaborative Learning, though further research is necessary to establish pedagogies.

1 Introduction

Beginning in the 1980s, academia has studied how to support and enhance Collaborative Learning (CL) in educational settings using technology. Referred to as Computer-Supported Collaborative Learning (CSCL), this pedagogical approach stems from social learning, an educational theory revolving around the idea that “new behavior can be acquired through the observation of other people’s behaviors” ( Shi et al., 2019 ) and focusing on social interaction between learners. CSCL’s strength appears to lie in its flexibility: by using characteristics of technology, both distant and face-to-face collaboration, as well as synchronous and asynchronous collaboration between learners, can be supported ( Stahl et al., 2006 ). As such, CSCL has been attributed numerous affordances, including joint information processing, sharing resources and co-construction of knowledge ( Shawky et al., 2014 ; Jeong and Hmelo-Silver, 2016 ).

An on-going development in the field of CSCL is the use of Virtual Reality, a technology that ‘[transports] a person to a reality (i.e., a virtual environment) which he or she is not physically present but feels like he or she is there’ ( Rebelo et al., 2012 ). These virtual environments (VEs) are shared, simulated spaces that allow distributed users to communicate with each other, as well as to participate in joint activities, making them an effective tool for remote collaboration ( Daphne et al., 2000 ). VEs tend to be highly customizable; their visual representation can be realistic (i.e., similar to reality or containing recognizable elements from reality) or abstract (e.g., three-dimensional representations of abstract concepts) depending on their purpose, making VEs adaptable for many different fields and disciplines ( Jackson et al., 1999 ; Joyner et al., 2021 ). Virtual Reality (VR), then, functions as a human-computer interface, allowing users to access these VEs through a variety of hardware, including flat-surface monitors and displays connected to desktop computers, room-sized devices called CAVE systems that project the VE onto its walls and Head-Mounted Displays (HMDs), helmets or headpieces that visualize the VE individually for each eye. In some cases, users inhabit avatars, virtual embodiments that represent their place inside the VE, though in other cases (such as the aforementioned CAVE systems, where users do not have to wear HMDs), no avatars are required for users to detect each other. Like VEs, the visual representation of avatars can be diverse: avatars can provide realistic depictions of users’ real-life appearances, but can also be visualized as something abstract, such as geometric objects or animals. Using these avatars to mediate interactions with each other, users progressively construct a shared understanding of the VE together ( Girvan, 2018 ). Of particular interest is VR’s ability to “immerse” users, providing them a sense of being inside the VE despite its non-physical, digital nature ( Freina and Ott, 2015 ). This immersion may lead to a state of presence, wherein users begin to behave inside the VE as they would in the physical world ( Jensen and Konradsen, 2018 ). Affordances of VR in education include enhancement of experiential learning ( Le et al., 2015 ; Kwon, 2019 ), spatial learning ( Dalgarno and Lee, 2010 ; de Back et al., 2020 ) and motivation and engagement among different types of learners ( Merchant et al., 2014 ; Chavez and Bayona, 2018 ). While research on VR has generally revolved around discovering its potential to support and enhance education, academics appear to agree that the field of educational use of VR lacks pedagogical practices or strategies, with little focus on how the technology should be implemented to reap its benefits ( Cook et al., 2019 ; Smith, 2019 ; Scavarelli et al., 2021 ).

VR technology has already shown potential for the field of CSCL, improving the effectiveness of team behavior, enhancing communication between group members and increasing learning outcome gains ( Le et al., 2015 ; Godin and Pridmore, 2019 ; Zheng et al., 2019 ). What makes the use of Virtual Reality for Collaborative Learning (VRCL) even more appealing for education is its diversity in hardware and, as a result, the different forms it can take depending on the setting. Whether learners interact with the VEs via display monitors, CAVE systems or HMDs, they all seem to produce positive effects such as positive learning gains and outcomes, as well as engagement and motivation for CL ( Abdullah et al., 2019 ; Zheng et al., 2019 ; de Back et al., 2020 ; Tovar et al., 2020 ).

To advance the field of VRCL, as well as to establish its benefits and affordances, several literature reviews have examined research on VRCL. For example, Muhammad Nur Affendy and Ajune Wanis (2019) , aiming to provide an overview of the capabilities of CL through the adoption of collaborative system in AR and VR, review how VEs are used for different types of collaboration (e.g., remote and co-located collaboration), with different VR hardware (e.g., eye tracking) and multiple intended uses (e.g., increasing social engagement and supporting awareness of collaboration among learners). In comparison, Zheng et al. (2019) evaluate VRCL technology affordances by conducting a meta-analysis as well as a qualitative analysis of VRCL prototypes to explore potential learning benefits; Scavarelli et al. (2021) explore a more theoretical side with the intention to produce educational frameworks for future VRCL-related research, discussing how several learning theories (e.g., constructivism, social cognitive theory and connectivism) are reflected in prior research on the potential of VR as well as Augmented Reality (AR) for social learning spaces.

Together, the literature reviews of Muhammad Nur Affendy and Ajune Wanis (2019) , Zheng et al. (2019) ; Scavarelli et al. (2021) describe a general optimism towards VR in educational settings to support collaboration. The reviews outline VRCL’s strengths as 1) its ability to enhance learning outcomes, 2) its potential to facilitate learning, 3) its effectiveness in supporting remote collaboration between learners, as well as experts and novices, 4) its support for interpersonal awareness between collaborating learners and 5) its diversity, both in terms of its customizability (allowing VEs to better suit objectives) as well as its technology. Affordances of VRCL are identified as 1) social interaction (strengthened by VR’s affordances of immersion and presence), 2) resource sharing (strengthened by VR’s ability to present imaginary elements) and 3) knowledge construction (supported by the two prior affordances of VRCL). Furthermore, challenges and gaps related to (research on) VRCL are outlined. First, accessibility should be considered a primary concern according to Scavarelli et al.,; this does not just relate to the technical accessibility of VR when used in education, but more so to the accessibility of social engagement between learners sharing these virtual learning spaces. Second, they recommend to explore the interplay and connectivity between VEs and the real world, as doing so could reveal new learning theories that innovate VRCL. Third, Zheng et al., suggest that research focus on pedagogical strategies involving VRCL, including how to apply VR to educational settings involving collaboration. Fourth, they propose a focus on finding a balance between using VRCL to recreate (or simulate) existing (“real”) situations and creating new situations that would normally be impossible, considering that prior work has primarily been centered on the former and as such misses out on VR’s potential to do the latter.

Considering that remote collaboration and distance learning, especially since the COVID-19 pandemic, are becoming increasingly important for learners, an understanding of VR’s potential for CL could prove beneficial for the field of education. While research on the topic is apparent, studies focusing on VR’s ability to support and enhance CL are still small in scale ( Zheng et al., 2019 ; Scavarelli et al., 2021 ), accentuating the scarcity of knowledge on the topic. This systematic review specifically centers on scientific research on VRCL, with a particular focus on the empirical knowledge that such literature has established. The aim of this paper is to examine in what ways VR supports and enhances CL according to prior research on these topics; to achieve this, it reports on what VRCL is used for in different fields of education, discusses what research has stated regarding VRCL in terms of affordances and benefits for education, describes the characteristics of VRCL that allow these benefits to come to fruition and provides an insight into the technology behind VRCL, as well as how this compares to the state-of-the-art of VR. In doing so, this study intends to identify possible gaps in the field of VRCL research for possible future studies, in addition to highlighting VRCL’s strengths to support current research. To the best of the authors’ knowledge, this study is the first systematic review on the topic of VRCL. As a means to provide the relevant information, this review addresses the following four research questions.

1. What skills and competences have been trained with use of VRCL (and what should a VRCL environment provide to train these)?

2. What domains and disciplines have been addressed (and why)?

3. What systems have been developed and/or established?

4. What empirical knowledge has been established (and with what methods and/or study designs)?

This section discusses the process of collecting the relevant studies for this literature review. In particular, the inclusion and exclusion criteria, databases and methods used are described.

2.1 Identification

The systematic review used two databases: Scopus and Web of Science. The search query contained the following key elements: 1) collaborative interaction, 2) VR, 3) education, training and learning, 4) simulations of a three-dimensional nature, 5) empirical data and 6) the use of a system (application or prototype). As such, the following search string was used in both databases:

[collaboration OR cooperation OR collaborative OR cooperative OR collaborate OR cooperate] [AND] ["virtual reality” OR “mixed reality” OR “extended reality"] [AND] ["3D” OR 3d OR 3-D OR 3-d OR threedimension* OR three-dimension* OR “three dimension*" OR CGI OR “computer generated” OR “computer-generated” OR model* OR construct*] [AND] [evaluat* OR data OR result* OR observ* OR empiric* OR trial* OR experiment* OR significan* OR participant* OR subject*] [AND] [education OR training OR learning OR university OR school OR vocational] [AND] [system* OR prototyp* OR application* OR program*]

To be considered suitable, papers had to meet five specific inclusion criteria. Firstly, an article had to discuss collaborative or cooperative interaction between human users of a virtual, three-dimensional simulation. Secondly, the article had to include and discuss Virtual-, Augmented-, Mixed Reality (MR) or Extended Reality (XR) as a three-dimensional simulation of a physical space or object(s). While this review focuses on VR for CL, mediums such as AR, MR and XR were included in this search for two reasons. On the one hand, definitions for these mediums appear to overlap to such an extent (with some even considering them too vague and ambiguous ( Tovar et al., 2020 )) that ‘pedagogical advantages of either technologies are [considered] comparable’ ( Sims et al., 2022 ). On the other hand, the mediums in question do not always get defined as separate ones, but rather as different points on one spectrum, commonly referred to as the virtuality continuum, in which ‘“reality” lies at one end, and “virtuality” […] at the other, with Mixed Reality […] placed between’ ( Scavarelli et al., 2021 ). As such, the decision was made to include these mediums, so as to ensure that no pedagogical advantages of VR would be excluded. The third inclusion criterium required an article to include an empirical study (i.e., containing qualitative or quantitative data) for it to be considered suitable. For the fourth and fifth criteria, an article had to contain an educational objective or goal (for human entities) and discuss a system used for educational purposes (for human entities) in order to be eligible.

Additionally, studies would be disqualified from the literature review if they 1) only described a patent, 2) only contained a summary (review) of a conference, 3) only consisted of a literature review, 4) were not accessible to the authors of this study, 5) were not available in English, 6) were a duplicate or a version, edition or release of an older study that already had been included or 7) did not specifically state the number of participants of any experiment involved in the study.

The search query resulted in 1,058 publications for Scopus and 845 studies for Web of Science, resulting in a total of 1,608 studies after duplicates were removed. Using the inclusion and exclusion criteria to filter out ineligible articles (initially based on title and abstract, then on full text), this review resulted in 139 articles analyzed. Results and details of the process (which followed the guidelines of the PRISMA method ( Moher et al., 2009 )) can be seen in Figure 1 . Appendix A shows the complete list of all 139 articles.

FIGURE 1 . Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram of the screening process.

To examine reliability of the selection process, five additional coders screened a random sample of 50 studies individually (10 per coder) using the inclusion and exclusion criteria. After comparing and discussing results, inter-rater reliability (between the first coder and the five coders) was calculated using a Kappa-metric, resulting in a moderate level of agreement of 0.77 ( McHugh, 2012 ) (results can be found in Supplementary Table B1 ).

A taxonomy ( Figure 2 ) was created to help classify all 139 articles. With this review’s research questions in mind, three vital topics were established to function as main categories for the coding process: education, system and evaluation (illustrated in column C1 in Figure 2 ). For RQ1 and RQ2, the first category, education, was established to extract information from the articles, concentrating on six classes. Similarly, information necessary to answer RQ3 was collected by coding attributes related to the second category, system, which included eight classes. Focusing the coding on elements related to the third category, evaluation (with five classes), allowed for extraction of relevant information required to answer RQ4. After the relevant categories, classes (visible in column C2 in Figure 2 ) and attributes (visible in column C3 in Figure 2 ) were decided upon, the classification hierarchy in Figure 2 was constructed, partially based on scientific literature ( Bloom et al., 1956 ; Schreiber and Asnerly-Self, 2011 ; Motejlek and Alpay, 2019 ), to provide assistance during the coding process. For an in-depth description of the motivation behind this classification hierarchy, please see Supplementary Appendix C . While the required information for some of these attributes could easily be inferred directly from each study, other attributes required the first coder to deduce which attributes were applicable.

FIGURE 2 . Classification hierarchy used for coding, including percent agreement ( p a ) and Cohen’s kappa (K) between first and second coder on the right.

To assess reliability of the first coder, a second coder classified articles with the taxonomy ( Supplementary Table D1, D2, D3 ). Inter-rater reliability between the two coders for 30 randomly selected studies was 0.60 (with a percent agreement of 0.85), considered a moderate level of agreement ( McHugh, 2012 ). Additionally, Figure 2 shows the inter-rater reliability for each individual class.

3 Descriptive results

In this section, discussion of descriptive results is divided into three sections according to the structure of the taxonomy. An overview of all results (according to the taxonomy) can be found in Figure 3 .

FIGURE 3 . Results of coding of data found in the literature, according to the taxonomy.

3.1 Education

As a first dimension, elements related to education were analyzed. A majority of the selected articles focused on VRCL in tertiary education (i.e., university), discussing possible uses for students. Educators providing support (e.g., scaffolding) for learners proved most prominent, though not all studies discussed this topic. While a wide selection of educational domains were discussed, computer sciences and social sciences were the most popular fields. Most studies specifically focused on synchronous collaboration. Prevalent among learning paradigms and educational approaches were problem-based learning (PBL) and constructivism. The specific results related to this dimension are found in Table 1 .

TABLE 1 . Distribution of Education-related attributes.

In contrast to the high number of articles focusing on tertiary education (64.0%), primary education was central in 10.8% while 5.0% discussed VRCL in secondary education. A small percentage of studies (6.5%) focused on types of learners outside of formal education (e.g., on-the-job training). In relation to the educators, a little over half of the studies reported on educators supporting the learners by providing varying degrees of scaffolding (55.4%). For 20.9% of cases, educators provided presentations and lectures inside the VE, providing a more passive learning experience. On a broader scope, the studies showed a wide variety of educational domains and fields of expertise to which VR was applied. While approximately a quarter of studies reviewed (25.9%) reported use of VRCL for education, specific domains that were often discussed included computer science, robotics, ICT and informatics (12.2%), social sciences (11.5%) medical fields (9.4%) and engineering (8.6%).

Also shown in Table 1 is the appearance of different types of social learning: 62.6% of studies reviewed discussed synchronous (collaborative) interaction, while in comparison a much lower 18.0% discussed asynchronous (cooperative) interaction. For a 10th of the studies, an expert-novice type of social learning was apparent (9.4%). On the topic of educational approaches and learning paradigms, 29.5% of articles did not seem to discuss any specific approaches. Among those that did, constructivism and PBL were featured substantially (33.1% and 41.0%, respectively), while paradigms such as experientialism, situated learning and distributed cognition were discussed less frequently. Other educational approaches, discussed in 35.3% of articles, included self-regulation and shared regulation (e.g., Al-Hatem et al., 2018 ) as well as cognitive apprenticeship (e.g., Bouta and Retalis, 2013 ). Looking at the learning goals and outcomes, the cognitive domain proved to be popular (50.4%), whereas affective and psychomotor domains were featured much less (7.9% and 5.0%, respectively). Other goals and outcomes included general student engagement (discussed in 31.7%) and support of collaboration amongst learners (60.4%).

The second dimension took a closer look at systems used in the studies, including aspects related to the hardware used (e.g., devices, types of control) as well as users’ interaction with VEs (e.g., degree of virtuality, virtual embodiment). A majority of the studies reviewed did not use VR technologies such as HMD-based VR (HMD VR), but instead focused on monitors and displays when discussing VRCL. Most studies chose general purpose controls (e.g., mouse and keyboard) over more advanced hardware such as positional tracking. A majority of studies provided their participants with full-body embodiment (e.g., avatars) and the ability to manipulate virtual objects while inside the VEs. Approximately a quarter of studies used systems for edutainment purposes (i.e., learning by having fun), while system use for training or therapeutic purposes was less common. Table 2 shows these results in detail.

TABLE 2 . Distribution of System-related attributes.

Results showed a clear preference for 3D (non-HMD) simulations, i.e., a virtual simulation of a (physical) environment projected on a surface or display that is not a Head-Mounted Display (and, as such, is considered less immersive): this degree of virtuality was far more prominent in the reviewed studies (78.4%) compared to the lesser implemented AR/MR (16.5%) and HMD VR (7.2%). The hardware used in these studies reflected this: a large amount (89.2%) implemented flat-surface monitors and displays to present VRCL environments. These studies commonly used desktop computer set-ups that included a keyboard, mouse and monitor, though in the case of AR and MR, surface-based mobile devices were often used. When using the system in a larger setting (i.e., larger group size), studies utilized projector-based (but still flat) surfaces to display the VE (e.g., Bower et al., 2017 ). In some cases, several types of these flat-surface displays were being used in different phases of a study (e.g., Nuñez et al., 2008 ). Cases that used CAVE systems (3.6%) included ImmersaDesks, CAVE-like devices that derive from the original CAVE systems. Studies that involved HMD VR used devices like the Oculus Rift and HTC Vive, while studies revolving around AR and MR implemented devices like the HoloLens. Some studies involved multiple devices to compare effects based on the difference (e.g., monitor-based vs HMD VR, as discussed in Vallance et al., 2015 ) while others discussed implementation of HMD VR and AR-related devices as possible future directions without using these in their experiments. With regard to user interaction, studies that implemented general purpose controls used simple computer keyboard and mouse, though some cases also involved video game controllers such as the Nintendo Wiimote and Nunchuck ( Li et al., 2012 ). Apart from the more default specialized controls such as 3DoF and 6DoF controllers or mobile device-based touch screens, studies also discussed a wide variety of other tools in this category, including multi-touch tabletops, haptic feedback devices, Xbox Kinect and gesture-sensing data gloves. While scarce, gaze control and positional tracking (15.1% and 11.5%, respectively) was primarily found in studies that used (mobile-based) AR and HMD VR, though some studies also provided these through devices such as the HoloLens or as part of a CAVE system.

Of the studies examined for this review, 55.4% discussed (self-developed) prototypes, while 44.6% used (pre-existing) applications. The most prominently-mentioned engine for prototypes was Unity, with % (of 77 studies) using it. Concerning the ones that used applications (62 of 139), more than half discussed VE application Second Life (%), while open-source VEs OpenSimulator and Open Wonderland were used in smaller numbers (% and %, respectively). In regard to the intended function of systems used, the majority of articles described a strictly educational one (58.3%) and revolved around implementing these systems in educational contexts as well as using them to facilitate collaborative learning. Studies that used systems to both educate and entertain (22.3%) tended to focus on game-based learning and serious games, though some cases also discussed video games originally not intended for educational purposes (e.g., World of Warcraft ( Kong and Kwok, 2013 ), Minecraft ( Mørch et al., 2019 )). When training purposes were mentioned (17.3%), this often indicated the use of VEs to train specific expertises, such as liver surgery or aircraft inspection. Rare cases where a system was used for therapeutic purposes (just 2.2%) included use of VRCL to teach social skills to patients with autism ( Ke and Lee, 2016 ) or to train physical activities amongst elderly ( Arlati et al., 2019 ).

Motivation behind studies’ choices for the size of collaboration differed between experimental reasons (e.g., a limited number of participants), pedagogical reasons (e.g., using pairs to better stimulate personal social interaction between members compared to larger groups) and reasons related to the systems (e.g., limited hardware availability). Small groups proved to be the most used group size, with 37.4% describing groups of between three and nine members. Pairs were used in 22.3% of studies. Motivations behind pairs included focus on expert-novice interaction and system capabilities (e.g., support for two users maximum). Articles that described larger groups (ten or more members) generally had entire classes of learners interact with system (15.1%).

Apart from a small number of studies that did not provide sufficient information on the matter, virtual embodiment of the users was featured prominently. In cases where physical attributes were virtually represented by (imagery of) tools (18.0%), the VRCL environment was often implemented for specific training of certain expertises. In general, partial virtual embodiment appears in first person, HMD VR (for example, when only the user’s hands are made visible); while scarce (3.6%), studies that displayed partial virtual embodiment provided some interesting examples outside of HMD VR. Examples of partial embodiment included a detailed 3D face to focus on emotional and social expressions ( Cheng and Ye, 2010 ) and using controllable, flat-surfaced rectangles in a 3D environment on which users’ real-life faces were projected via webcam ( Nikolic and Nicholls, 2018 ). Full-body embodiment proved to be the most popular, with 67.6% of studies using systems that provide users complete (full-body) virtual representation. To a degree, the relatively high number of studies that present full-body embodiment can be explained by the systems that were implemented; applications such as Open Simulator and Second Life provide users with customizable avatars, making a full-body virtual embodiment a default feature. In some cases, however, studies specifically examined the effects of virtual embodiment, such as Gerhard et al. (2001) examining possible influences of different avatars on users’ sense of presence. On the topic of user influence on VEs, a little more than half the studies (53.2%) used systems that allowed (some degree of) virtual object manipulation, whereas approximately a quarter of the studies (26.6%) also provided users the tools to manipulate actual content of the VRCL environment. In 16.5% of studies, the system only allowed users to be visibly present inside the VE, while only 3.6% did not provide sufficient information on the matter.

3.3 Evaluation

For the third dimension, the selected articles were analyzed on how they evaluated applying VRCL. Articles frequently concentrated on evaluation of the system(s), with a higher number of them using self-report evaluation methods. Study design of the studies shows a similar result: pre-experimental study design (typically used for preliminary testing of systems) was regularly implemented, with surveys being a popular method of collecting data. While the number of participants was diverse, roughly half of studies reviewed used a sample size between 1 and 25 participants. The majority of articles discussed positive outcomes, whereas only a small amount featured negative results. Detailed results are displayed in Table 3 .

TABLE 3 . Distribution of Evaluation-related attributes.

The majority of studies focused on evaluating a system’s effectiveness when using it in educational settings (71.2%). These studies concentrated on the system’s capacity to support collaboration between learners. Other topics of discussion were student interest in the system and how the system can facilitate learning. Whenever studies examined processes (34.5%), evaluation would be centered around attempts to understand how group interaction materializes in these environments. This included how learners resolve social conflicts ( Cheong et al., 2015 ) and examining how co-presence (e.g., Kong and Kwok, 2013 ) and PBL take shape in VRCL environments. 35.3% of articles discussed learning outcomes after participants interacted with the system. The few situations where the above three attributes did not apply (3.6%) included a study that aimed to develop design guidelines ( Economou et al., 2001 ) and a study primarily interested in the teacher’s role when learners interact with VEs ( Lattemann and Stieglitz, 2012 ).

Most studies collected self-reported data from their participants (85.6%), while over half used behavioral methods to obtain tracking and observational data (59.0%). Articles that reported on knowledge- and/or performance-based assessments (20.9% of studies) often used pre- and post-tests to acquire their data, while only one appeared to use physiological data, tracking participants’ heart rate (0.7%). A notable number of articles (79.9%) implemented pre-experimental design in their studies. Some of these were case studies, applying VEs to educational settings (e.g., Terzidou et al., 2012 ), while others performed pilot studies to establish a first impression of the effects of a system on specific pedagogical situations (e.g., examining how VE-based application OpenSimulator influences Transactive Memory Systems amongst learners ( Kleanthous et al., 2016 )). Quasi-experimental- (13.7%) and true experimental designs (5.8%) were used scarcely, while only 2 out of 139 studies (1.4%) performed an experiment with single-subject design. With respect to non-experimental and descriptive designs, 84.9% of studies implemented a survey-based design, whereas a little over half used observational designs to collect data (56.1%). In some cases, comparative and correlation designs were implemented (7.9% and 15.8%, respectively).

Table 3 also reveals that approximately half of the studies sampled between 1 and 25 participants (53.2%), while around a quarter (26.6%) used a sample size between 26 and 50 participants. For 13.7% of articles, between 51 and 100 participants were used, whereas only 6.5% discussed using more than 100 participants for collecting data. In terms of outcomes, around half of the studies concluded that their system(s) seemed positive and promising (53.2%), while 17.3% draw positive conclusions based on significant outcomes from statistical hypothesis testing. Negative outcomes were scarce, with only 2.2% of the studies reporting negative results. Mixed outcomes were reported for 7.2% of the studies, whereas 20.1% discussed results that were inconclusive, showed no effect or reported outcomes on which positive and negative effects are not applicable.

4 Qualitative results

In general, the literature reviewed for this paper shows a positive attitude towards the use of VR to support and enhance CL. However, the results quickly make it apparent that the methods of applying VR to educational fields to support and enhance CL can vary greatly amongst the studies examined here. In order to acquire a general understanding how these studies have attempted to support and enhance CL using VR, this section will discuss qualitative results established. The rest of this section will be divided into sub-sections, each focusing on discussing results related to one of the four research questions of this literature review.

4.1 Skills and competences trained with VRCL

A number of elements can be identified regarding skills and competences trained with VRCL. Based on the skills and competences discussed in the reviewed literature, five categories were established for this study with the intention to provide a concise overview. These categories, including examples of each category, can be viewed in Table 4 .

TABLE 4 . Skills focused on in the reviewed literature.

For the types of skills and competences shown in Table 4 to be trained effectively, a VRCL environment requires a number of features that support the learners in learning these abilities. Based on the information provided by the reviewed literature, nine required features and design parameters of VRCL can be identified. First, virtual embodiment plays an important role in how learners view themselves and each other inside the VE, impacting learning outcomes and collaborative behavior by providing a sense of awareness and belonging ( Edirisingha et al., 2009 ; McArdle and Bertolotto, 2012 ). Second, efficient communicational tools are essential for effective collaboration: verbal (audio) communication is crucial ( Economou et al., 2001 ; De Pace et al., 2019 ), though additional modalities such as haptic technology can further enhance collaboration ( Moll and Pysander, 2013 ). Third, usability and accessibility should be taken into consideration: VRCL systems should be accessible to all levels of technical skills as differences negatively affect group cohesion and learning between group members (Y. Chang et al., 2016 ; Denoyelles and Kyeong-Ju Seo, 2012 ). Fourth, learners’ perceived usefulness of the VE also affects group cohesion; factors such as awareness, presence and social presence appear to significantly influence this perceived usefulness ( Denoyelles and Kyeong-Ju Seo, 2012 ; Yeh et al., 2012 ). Fifth, the ability to interact with elements inside the VE are considered key: to optimize learning outcomes, learners must have the option to manipulate elements inside the VE (e.g., virtual objects or virtual tools) in a seemingly natural and intuitive way ( Vrellis et al., 2010 ; Bower et al., 2017 ). Sixth, academic efficacy can be achieved if tasks inside the VE are designed around its educational, collaborative objectives, especially when designed for equal input from all learners in a group ( Wang et al., 2014 ; Nisiotis and Kleanthous, 2019 ). Seventh, educators should be ready to provide support, motivation and moderation of collaboration while learners interact inside the VE ( Lattemann and Stieglitz, 2012 ; Bower et al., 2017 ). However, the eighth feature, a level of autonomy, is equally important for each individual learner, not just in terms of independence from the educators, but more importantly from each other, as this allows them to provide different points of views as well as to explore multiple representations, thus improving CL ( Hwang and Hu, 2013 ). Ninth, implementation of VRCL should make sure to primarily support socialization inside the VE, as underestimating the importance of socialization might lead to features of VR obstructing rather than facilitating CL ( Chang et al., 2009 ).

Surprisingly, only a small number of the literature reviewed focused on goals related to the affective domain (7.9%). With some calling VR the “ultimate empathy machine” ( Rueda and Lara, 2020 , p.6), the medium’s ability to induce emotions has been prominently discussed and studied. Not only has VR been shown to indeed be capable of enhancing empathy amongst users ( Herrera et al., 2018 ), with some even arguing it to be more effective than traditional empathy-shaping methods ( Liu, 2020 ), studies have also suggested it to be an effective tool to offer a uniquely different level of understanding ( de la Peña et al., 2010 ). This would suggest that VR’s ability to create a better understanding of different group members’ points of view could in turn support collaboration between learners.

Similarly, even less literature reviewed focused on goals related to the psychomotor domain (5.0%). Prior studies have been positive and hopeful regarding VR to expand the possibilities of physical training ( Pastel et al., 2020 ). Interestingly, technical features such as positional tracking even seem to be effective in predicting psychomotor outcomes ( Moore et al., 2021 ), which could prove useful for domains that specifically focus on expert-novice training in primarily physical tasks (e.g., certain types of engineering). However, positional tracking, not unlike psychomotor outcomes, is only discussed sparingly (11.5%) in the literature reviewed.

An interesting observation in relation to the evaluation methods used in the scientific literature is that only 1 out of 139 articles used physiological measures. As suggested by research, physiological synchrony between group members can serve as an effective indicator for the quality of interpersonal interaction between them (with a higher physiological synchrony correlating with a higher interaction level) ( Liu et al., 2021 ). Furthermore, physiological measurements can be used to identify multiple predictors related to education and training, including the quality of collaboration between group members ( Dich et al., 2018 ). Additionally, visualizing physiological results of each member of a group to the others in real-time during collaboration has shown to have a positive effect on the empathy levels and cohesion of the group, further suggesting how collaboration between learners could benefit from physiological measures ( Tan et al., 2014 ). Considering VR’s visual characteristics as well as research arguing that physical signals such as electroencephalogram (EEG) can conveniently and unobtrusively be tracked during use of HMD VR ( Tremmel et al., 2019 ), future research on VRCL could prove fruitful in terms of training collaborative skills and competences via use of physiological-based information.

4.2 Disciplines focused on regarding VRCL

When looking at the most prominently-featured domains in the literature reviewed (as shown in Figure 4 ), examining what motivated researchers to study VRCL in the field of 1) education, 2) computer science, robotics and informatics and 3) social sciences can provide an understanding of VRCL’s role in these different disciplines.

FIGURE 4 . Results of the educational disciplines focused on in the reviewed literature.

For the field of education, some studies focus on the potential behind VRCL, intending to discover what it can mean for the development of cognitive and technical skills ( Franco and De Deus Lopes, 2009 ). Other studies focus on possible learning gains, examining how knowledge gained in VEs transfers to the real world (i.e., how learners apply outcomes in VEs to situations in actual reality) or attempting to facilitate this transfer by implementing elements of both ( Carron et al., 2013 ). In certain cases, articles specifically examine VEs’ effects on collaboration and how VR can be used to reinforce CL (e.g., Tüzün et al., 2019 ), whereas others aim to determine if existing educational paradigms such as constructivism can be applied to VRCL environments and, if so, how that affects group knowledge gain between learners ( Girvan and Savage, 2010 ). Together, these studies present a general motivation to discover what VRCL can mean for education and where its potential may lie.

For computer science, robotics and informatics, use of VRCL can be summarized in two motivations: 1) innovate these domains and 2) create a learning community. In the first case, researchers intend to utilize the affordances VRCL environments have to offer to further advance fields such as computer science, which have been criticized in the past for using two-dimensional learning platforms and oral-based teaching methods ( Pellas, 2014 ). With VEs, educators can provide learners realistic yet illusionary worlds that are flexible, customizable and even allow for detailed statistics on learners’ performance ( Champsas et al., 2012 ). In the second case, reviewed articles vocalize a desire to use VRCL to provide learners purposeful collaborative activities that create a sense of belonging to a learning community, using aspects such as awareness, presence and different methods of communication to motivate learners in these fields to work together closely ( De Lucia et al., 2009 ).

In similar fashion, social studies appears to be interested in how socialization between learners is manifested inside VRCL (e.g., Edirisingha et al., 2009 ). Some articles go further, studying how VRCL can support socialization: Molka-Danielsen and Brask (2014) suggest that presence, awareness and belonging allow for communication, negotiation and trust between learners, elements deemed necessary for completing collaborative tasks. Other studies focus on specific characteristics of socialization, such as how gender could affect social interaction and group cohesion inside VEs ( Denoyelles and Kyeong-Ju Seo, 2012 ). Collectively, these articles show a desire to understand how elements related to socialization transfer to VRCL, as well as how these environments can sustain and even enhance those elements.

4.3 Systems developed and/or established for VRCL

The results related to systems used show that there is quite a disparity between use of HMD VR and that of non-HMD VR. Almost 80% of systems implemented non-HMD VR, with AR/MR and HMD VR implemented far less frequently (16.5% and 7.2%, respectively, as illustrated in Figure 5 ). Almost 90% of studies described the use of flat-surface monitors and displays, which, when compared to the 10.8% of studies that used HMD devices, further highlights the low use of HMD VR in the literature reviewed (see Figure 6 ).

FIGURE 5 . Results of the degree of virtuality of systems discussed in the reviewed literature.

FIGURE 6 . Results of the hardware used in the reviewed literature.

The lack of representation of HMD VR in these articles is somewhat surprising, considering this type of virtuality and hardware is commonly associated with the medium of VR ( Dixon, 2006 ; Bonner and Reinders, 2018 ; Jing et al., 2018 ). The statement that research into application of VR to the field of education lacks a focus on HMD VR, however, is not uncommon ( Sousa Santos et al., 2009 ; Scavarelli et al., 2021 ), thus begging the question: why is it underrepresented in the reviewed literature?

One possible explanation could be that HMD VR is known to be difficult to apply to educational settings because of its high costs ( Olmos et al., 2018 ). Some of the articles analyzed for this review were published in the late 90s; while HMD VR technology was already available in those times, devices were more expensive and less technologically advanced compared to the technology that is available now ( Mehrfard et al., 2019 ; Wang et al., 2022 ). Furthermore, the technical skills necessary to implement VR properly in educational settings can prove challenging ( Jensen and Konradsen, 2018 ). Since collaboration involves multiple people, difficulties related to accessibility could be more severe when applying VR to a larger group of learners. Another possible reason is the health risks associated with the technology: HMD VR is often connected to motion sickness and cybersickness ( Olmos et al., 2018 ; Yoon et al., 2020 ). A third reason refers to the general lack of pedagogy on the topic of HMD VR: while the medium’s potential for education is often discussed, general guidelines as to how it should be applied efficiently to educational settings ( Cook et al., 2019 ; Zheng et al., 2019 ) as well as an understanding of how learning mechanisms operate inside VR environments ( Smith, 2019 ) are missing. Naturally, the small size of research done on VR and CL exacerbates this lack even further when specifically discussing VRCL. A possible fourth reason that is more closely tied to this particular literature review is that, despite its popularity in research, HMD VR appears to still lack empirical evidence of its educational value ( Sousa Santos et al., 2009 ; Makransky et al., 2019 ; Radianti et al., 2020 ), which, considering this review’s focus on empirically-based knowledge, could explain its scarcity.

The low representation of HMD VR and high representation of non-HMD VR could be related to the ongoing discussion about what defines VR and how it differs from VEs, as discussed in-depth by Girvan, (2018) . Girvan argues that some use terms synonymously with VR and/or VEs, while others use these same terms to classify different types of VEs, thus creating a fragmented understanding of what these are (and what they are not). Girvan’s point is reflected in the reviewed literature of this paper: while some studies identify Second Life as a “virtual environment” or “virtual world” (e.g., Terzidou et al., 2012 ), others refer to it as “virtual reality” (e.g., Sulbaran and Jones, 2012 ). To prevent further confusion with technologies with similar technical features, Girvan suggests to conceptualize VEs as ‘shared, simulated spaces which are inhabited and shaped by their inhabitants, who are represented as avatars [that] mediate our experience of this space as (…) we interact with others, with whom we construct a shared understanding of the world at that time’. VR, then, should be defined as ‘a technical system through which a user or multiple users can experience [such] a simulated environment’ ( Girvan, 2018 ).

Apart from causing a fragmented understanding of the terms in the literature, different interpretations of VR and VEs also lead to HMD VR and non-HMD VR being described as one and the same thing under the moniker of “virtual reality”. Though this may seem a trivial dispute about labels, treating these two types as identical will lead to misconceptions regarding both, as HMD and non-HMD VR contain different benefits and limitations when applied to education. While some studies showed no differences between the two in terms of specific learning outcomes (e.g., spatial- ( Srivastava et al., 2019 ) and language learning (J. Y. Jeong et al., 2018 )), other research highlighted several differences between HMD and non-HMD. Compared to non-HMD, HMD VR has shown to provide a much higher sense of embodiment, which in turn is hypothesized to lead to higher performances, in particular in psychomotor skills ( Juliano et al., 2020 ; Saldana et al., 2020 ). Similarly, HMD VR appeared superior to computer screens in terms of arousal, engagement and motivation in learners ( Makransky and Lilleholt, 2018 ). In contrast, however, Makransky et al. (2019) reported overloads and distractions caused by HMD VR, leading to poorer learning outcomes compared to non-HMD, a sentiment shared by Parong and Mayer (2021) , who described HMD VR to cause high affective and cognitive distractions. Amati and McNeill (2012) even argue that the difference between HMD and non-HMD VR (and in particular how the two are interacted with by users) have severe implications for teaching and practice.

With all of the above in mind, the low representation of HMD VR in the literature examined for this review can be interpreted in two ways. On the one hand, the underutilization underlines that HMD VR is not being used to its full potential and could very well hold much more promise for the field of education and CL. On the other hand, the low use of HMD VR could suggest that implementation of HMD VR in education and/or CL is, in fact, not worth the trouble it brings with it. Whether HMD VR is a benefit or a burden, then, arguably depends on three important elements: 1) the goals (i.e., what skills and/or competences are supposed to be trained), 2) the setting (i.e., the disciplines and fields to which it is applied), and 3) the affordances of VRCL (and to what degree these conform to the goals and setting).

4.4 Empirical knowledge established regarding VRCL

When summarizing the outcomes of the 139 articles, 70% of the studies reviewed displayed a positive attitude towards the application of VRCL to education. While a relatively low number (approximately 25%) presented statistically significant outcomes, this does illustrate a strong optimism amongst those studying VRCL environments in different fields of education as described in prior literature reviews on the topic. This could also explain the high number of studies that deployed pre-experimental study designs: with VRCL being a relatively new addition to the world of CSCL, as well as one that continues to rapidly advance because of the technology behind it, many seem enthusiastic and eager to see what promises VRCL holds when used in different fields and with different types of learners.

Regarding affordances discussed in the reviewed literature, several features are identified. First, VRCL appears an efficient tool to engage learners and to motivate them to study and learn. The ability to customize VRCL environments and their content provides learners more personalized experiences that better suit their personalities and attitudes, thereby enhancing the motivation to learn on both an individual and group level ( Arlati et al., 2019 ). Furthermore, VRCL’s immersive qualities tend to make the experiences more engaging for learners, encouraging them to engage in presentations and demonstrations as well as to communicate and collaborate with each other ( Avanzato, 2018 ).

The second affordance identified VRCL as a great tool for distance learning and remote collaboration. VEs provide a method for learners and educators to work together and collaborate despite distances. In comparison to other media, however, VRCL brings with it a high sense of immediacy (i.e., ‘verbal and non-verbal behaviors that give a sense of reduction of physical and psychological distance between the communicators’), which in turn presents an increased perception of learning ( Edirisingha et al., 2009 ). Additionally, VRCL’s immersive qualities and high presence allow for environments capable of simulating training as preparation for real-life experiences ( Al-Hatem et al., 2018 ) that simultaneously promote active participation and social interaction ( Mystakidis et al., 2017 ) in a setting that feels personal despite distances between learners ( Desai et al., 2017 ). In certain cases, such as education for learners with physical disabilities, learners and educators even considered connectivity to be more accessible and easier than real-life equivalents ( Aydogan and Aras, 2019 ), illustrating that VRCL environments can potentially go beyond simply being a replacement. To effectively support the distance learning and remote collaboration, however, design of the VEs should focus on providing learners a sense of 1) presence, 2) awareness and 3) belonging to the group ( Molka-Danielsen and Brask, 2014 ).

Thirdly, the literature reviewed suggests that VRCL environments are effective spaces to support multi- and interdisciplinary learning and collaboration. The ability to customize VEs, adapting to suit users’ needs, prevents them from being restricted to just a single specific subject field. This in turn allows educators to change the environments to accommodate many different subject fields and topics so as to make sure that learners from different backgrounds can collaborate with each other undisturbed ( Bilyatdinova et al., 2016 ). Moreover, it seems that VRCL environments made some of the literature studies reviewed realize the importance of interdisciplinary collaboration in the learning process ( Franco et al., 2006 ; Nadolny et al., 2013 ).

The fourth affordance identified might be an unsurprising but nonetheless important one: VRCL seems to be a tool for the development of social skills. While identity construction and projection through virtual embodiments can be complex for learners (depending on their technical skills), VRCL is found to facilitate social presence and foster socialization ( Edirisingha et al., 2009 ). VRCL’s customizability allows learners to integrate personal preferences and identity expressions into processes inside the environment (e.g., through their virtual embodiments), in turn mediating identity and norm construction for real-life social settings ( Ke and Lee, 2016 ). Vital social skills, such as the ability to identify and manipulate basic emotional states, can be taught and trained using VEs, improving learners’ socialization, communication skills and emotional intelligence ( López-Faican and Jaen, 2020 ). Learners’ prior experience with VEs, however, should not be underestimated, as a difference in familiarity with VRCL environments has been shown to impact collaboration ( Bluemink et al., 2010 ).

Fifth, VEs appear fitting for CL-related learning paradigms and educational approaches. Some studies specifically focus on examining to what degree VRCL environments are applicable to paradigms such as constructivism, socio-constructivism and constructionism (e.g., Girvan and Savage, 2010 ; Pellas et al., 2013 ; Abdullah et al., 2019 ), concluding that these indeed go well together. Other studies, however, focus on theories and methods commonly associated with these paradigms. In particular, experiential learning and PBL seem appropriate for VRCL environments. VEs allow for safe, consequence-free learning for exploring, experiencing and practicing without any real-life risks ( Cheong et al., 2015 ; Le et al., 2015 ), making it suitable for experiential learning. Moreover, VRCL’s immersive qualities seem to support and even elevate experiential learning strategies such as roleplay and improvisation, providing learners close to real-world experiences in a controlled environment ( Jarmon et al., 2008 ; Ashley et al., 2014 ). In the case of PBL, each individual learner can use different tools inside VRCL environments to illustrate and represent ideas and suggestions to the rest of the group. Considering that VEs seem great tools for conceptual learning because of their customizability and visual nature ( Brna and Aspin, 1998 ; Griol et al., 2014 ), learners can use these features to explain their point of view in ways that they otherwise could not. As a result, learners appear to become more active and effective in sharing ideas, joint problem solving and the co-construction of mental models when working in groups inside VRCL environments ( Rogers, 2011 ; Hwang and Hu, 2013 ).

Returning to the topic of disparity between HMD and non-HMD VR represented in the reviewed literature, as well as both being discussed as one and the same “Virtual Reality”, an important question to ask is whether the affordances identified here are transferable between the two. HMD and non-HMD VR differ in several ways: they are interacted with differently, face different obstacles when applied to education and appear to have different learning outcomes based on different educational settings.

With the definitions of VEs and VR as given by Girvan (2018) as a frame of reference, however, an answer can be given regarding the transferability of these affordances between HMD and non-HMD VR. Both HMD and non-HMD VR should be considered tools, technical systems through which users can virtually enter VEs, i.e., shared simulated spaces in which they can interact with the environment as well as each other. As such, the affordances described in this paper do not revolve around the tools used, but that which they provide access to: the VRCL environments. Simultaneously, which tool is used to access these VRCL environments can in turn affect both the interaction and the outcome of users’ experiences with VEs. For example, HMD VR might offer more effective development of social skills compared to non-HMD VR, considering the former provides a higher sense of embodiment and, in extension, more intuitive and expansive methods of expression. If, however, cognitive learning outcomes are the most important educational objective, non-HMD VR could be a better option, considering HMD VR’s tendency to cause affective and cognitive distractions. This, then, reflects the aforementioned statement regarding HMD VR being a benefit or a burden. While affordances of VRCL environments apply to both HMD and non-HMD VR, the effect of these affordances depend on 1) the goals, 2) the setting and 3) which affordances of VRCL are most vital to the first two elements. As such, the choice between non-HMD VR and HMD VR should be made depending on those three elements.

5 Conclusion and future research

With current research on the topic being scarce while the demand for remote collaboration and distance learning keeps increasing, this literature review intends to study how VR has been (and can be) used to support and enhance CL. To achieve this, it attempts to answer four research questions regarding prior research on VRCL: what skills and competences have been trained with VRCL and what does VRCL provide in these scenarios? To what educational domains has VRCL been applied and why? What systems have been used for VRCL? And what empirical knowledge has been established regarding VRCL?

This paper identifies five types of skills and competences commonly trained with the use of VRCL. Furthermore, a number of features and design principles are identified in terms of what these environments should offer for these skills to be developed. Educational fields and domains appear to be interested in VRCL because of a desire to innovate, to form communities, to support remote collaboration and to enhance socialization skills of learners. In terms of technology, systems used for VRCL-related purposes appear to predominantly focus on monitor-based (non-HMD) VR and mouse-and-keyboard controls, contrasting what VR is commonly associated with (e.g., HMD VR, specialized controls involving gaze control and positional tracking). This study perceives a general optimism present in the literature reviewed regarding the use of VR to support and enhance CL in learners. Additionally, a number of affordances of VRCL are described, though it is of importance to note that these affordances could differ in strength depending on which type of VR (i.e., non-HMD or HMD) is used.

While the literature on VRCL reviewed for this paper is diverse, it suggests that Virtual Reality can be an effective tool for supporting and enhancing Collaborative Learning. This diversity, however, also highlights that pedagogies of VRCL are lacking, with studies showing many different and contrasting approaches to applying VR to their respective fields for the support of CL. In order to see VR become more adopted as an educational tool for collaborative purposes, pedagogies should be clearly structured, highlighting similarities and differences in regards to both the technologies used and the domains they are used in. As such, this paper proposes a number of suggestions for future research. First, the difference between hardware used in the literature reviewed and the state-of-the-art of VR suggests that further examination of differences between non-HMD and HMD VRCL, both in terms of affordances as well as challenges and obstacles, could lead to a better understanding of VRCL’s potential. Second, despite the advantages VR has for development in affective and psychomotor skills, the scientific literature on VRCL shows only minor focus on these domains. This study argues that CL would benefit from both these domains being featured more prominently and as such encourages more research into these matters. Third, this paper suggests that research into VRCL focuses on using study designs and evaluation methods that are less frequently (or barely) featured in the reviewed literature. While the repeated and dominant use of pre-experimental study design is understandably meant to identify the potential behind the technology, the domain of VRCL (and, in extension, research on VR in education) would benefit from more true experimental design. Additionally, considering that the use of physiological data for evaluation methods appears to be unexplored terrain, this paper suggests that future research into VRCL implements these types of methods.

Author contributions

NvM: Main author VvW: Co-author and coder W-PB: Co-author and supervisor MS: Co-author and supervisor. All authors contributed to the article and approved the submitted version.

This project has been funded by the Leiden-Delft-Erasmus Centre for Education and Learning (LDE-CEL).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2023.1159905/full#supplementary-material

Supplementary Appendix A | List of all articles included.

Supplementary Appendix B1 | Results of agreement between first author and five additional coders on in- and exclusion criteria.

Supplementary Appendix C | Explanation/motivation behind Taxonomy.

Supplementary Appendix D1 | Results of agreement between first author and coder on use of taxonomy’s first category (Education).

Supplementary Appendix D2 | Results of agreement between first author and coder on use of taxonomy’s second category (System).

Supplementary Appendix D3 | Results of agreement between first author and coder on use of taxonomy’s third category (Evaluation).

Abdullah, J., Mohd-Isa, W. N., and Samsudin, M. A. (2019). Virtual reality to improve group work skill and self-directed learning in problem-based learning narratives. Virtual Real. 23 (4), 461–471. doi:10.1007/s10055-019-00381-1

CrossRef Full Text | Google Scholar

Al-Hatem, A. I., Masood, M., and Al-Samarraie, H. (2018). Fostering student nurses’ self-regulated learning with the Second Life environment: An empirical study. J. Inf. Technol. Educ. Res. 17, 285–307. doi:10.28945/4110

Amati, M., and McNeill, M. (2012). Learning from and through virtual worlds: A pilot study of second life. J. Educ. Built Environ. 7 (1), 39–55. doi:10.11120/jebe.2012.07010039

Arlati, S., Colombo, V., Spoladore, D., Greci, L., Pedroli, E., Serino, S., et al. (2019). A social virtual reality-based application for the physical and cognitive training of the elderly at home. Sensors Switz. 19 (2), 261. doi:10.3390/s19020261

Ashley, C., Kibbe, S., and Thornton, S. (2014). Experiential learning in second life: An application in retail management. Atl. Mark. J. 3 (2), 94–112. http://conf.vwbpe.org/index.php/VWBPE/11/paper/view/132 .

Google Scholar

Avanzato, R. (2018). Virtual technology to support student engagement and collaboration in online engineering courses. Comput. Educ. J. 9 (4). https://www.scopus.com/inward/record.uri?eid=2-s2.0-85058935867&partnerID=40&md5=c38a848de46d17a869b0307c6046d06e .

Aydogan, H., and Aras, F. (2019). Design, simulation and virtual implementation of a novel fundamental programmable logic controllers laboratory in a 3D virtual world. Int. J. Electr. Eng. Educ. 59, 266–281. doi:10.1177/0020720919856249

Bilyatdinova, A., Karsakov, A., Bezgodov, A., and Dukhanov, A. (2016). Virtual environment for creative and collaborative learning. Adv. Intelligent Syst. Comput. 416, 371–381. doi:10.1007/978-3-319-27478-2_26

Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., and Krathwohl, D. R. (1956). “The classification of educational goals,” in Taxonomy of educational objectives (New York, NY: David McKay ), 62–197.

Bluemink, J., Hämäläinen, R., Manninen, T., and Järvelä, S. (2010). Group-level analysis on multiplayer game collaboration: How do the individuals shape the group interaction? Interact. Learn. Environ. 18 (4), 365–383. doi:10.1080/10494820802602444

Bonner, E., and Reinders, H. (2018). Augmented and virtual reality in the language classroom: Practical ideas. Teach. Engl. Technol. 18 (3), 33–53. http://www.tewtjournal.org .

Bouta, H., and Retalis, S. (2013). Enhancing primary school children collaborative learning experiences in maths via a 3D virtual environment. Educ. Inf. Technol. 18 (4), 571–596. doi:10.1007/s10639-012-9198-8

Bower, M., Lee, M. J. W., and Dalgarno, B. (2017). Collaborative learning across physical and virtual worlds: Factors supporting and constraining learners in a blended reality environment. Br. J. Educ. Technol. 48 (2), 407–430. doi:10.1111/bjet.12435

Brna, P., and Aspin, R. (1998). Collaboration in a virtual world: Support for conceptual learning? Educ. Inf. Technol. 3 (3–4), 247–259. doi:10.1023/a:1009649631868

Carron, T., Pernelle, P., and Talbot, S. (2013). “Issues of learning games: From virtual to real,” in IADIS international conference on cognition and exploratory learning in digital age, CELDA (Texas, TX: International Assn for Development of the Information Society (IADIS) ), 133–140.

Champsas, I., Leftheris, I., Tsiatsos, T., Terzidou, T., and Mavridis, A. (2012). “Open games: A framework for implementing 3D collaborative educational games in opensim,” in Proceedings of the European conference on games-based learning (Cork, Ireland: European Conference on Games Based Learning (ECGBL) ).

Chang, V., Gütl, C., Kopeinik, S., and Williams, R. (2009). Evaluation of collaborative learning settings in 3D virtual worlds. Int. J. Emerg. Technol. Learn. 4, 6–17. doi:10.3991/ijet.v4s3.1112

Chang, Y., Aziz, E. S. S., Zhang, Z., Zhang, M., and Esche, S. K. (2016). Usability evaluation of a virtual educational laboratory platform. Comput. Educ. J. 16 (1), 24–36. doi:10.18260/p.24973

Chavez, B., and Bayona, S. (2018). “Virtual reality in the learning process,” in Advances in intelligent systems and computing (Midtown Manhattan, New York City: Springer International Publishing ), 746. doi:10.1007/978-3-319-77712-2_129

Cheng, Y., and Ye, J. (2010). Exploring the social competence of students with autism spectrum conditions in a collaborative virtual learning environment - the pilot study. Comput. Educ. 54 (4), 1068–1077. doi:10.1016/j.compedu.2009.10.011

Cheong, Y. G., Khaled, R., HolmgȦrd, C., and Yannakakis, G. N. (2015). “Serious games for teaching conflict resolution: Modeling conflict dynamics,” in Conflict and multimodal communication: Social research and machine intelligence (Cham, Switzerland: Springer ), 449–475. doi:10.1007/978-3-319-14081-0_21

Cook, M., Lischer-Katz, Z., Hall, N., Hardesty, J., Johnson, J., McDonald, R., et al. (2019). Challenges and strategies for educational virtual reality: Results of an expert-led forum on 3D/VR technologies across academic institutions. Inf. Technol. Libr. 38 (4), 25–48. doi:10.6017/ital.v38i4.11075

Dalgarno, B., and Lee, M. J. W. (2010). What are the learning affordances of 3-D virtual environments? Br. J. Educ. Technol. 41 (1), 10–32. doi:10.1111/j.1467-8535.2009.01038.x

Daphne, E., William, M. L., Steve, P. R., and Adrian, W. J. (2000). “CVE technology development based an real world application and user needs,” in Proceedings IEEE 9th International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WET ICE 2000) , Gaithersburg, MD, USA , 14-16 June 2000 . doi:10.1109/ENABL.2000.883698

de Back, T. T., Tinga, A. M., Nguyen, P., and Louwerse, M. M. (2020). Benefits of immersive collaborative learning in CAVE-based virtual reality. Int. J. Educ. Technol. High. Educ. 17 (1), 51. doi:10.1186/s41239-020-00228-9

de la Peña, N., Weil, P., Llobera, J., Giannopoulos, E., Pomés, A., Spanlang, B., et al. (2010). Immersive journalism: Immersive virtual reality for the first-person experience of news. Presence Teleoperators Virtual Environ. 19 (4), 291–301. doi:10.1162/PRES_a_00005

De Lucia, A., Francese, R., Passero, I., and Tortora, G. (2009). Development and evaluation of a system enhancing second Life to support synchronous role-based collaborative learning. Softw. - Pract. Exp. 39 (12), 1025–1054. doi:10.1002/spe.926

De Pace, F., Manuri, F., Sanna, A., and Zappia, D. (2019). A comparison between two different approaches for a collaborative mixed-virtual environment in industrial maintenance. Front. Robotics AI 6, 18–14. doi:10.3389/frobt.2019.00018

Denoyelles, A., and Kyeong-Ju Seo, K. (2012). Inspiring equal contribution and opportunity in a 3d multi-user virtual environment: Bringing together men gamers and women non-gamers in Second Life®. Comput. Educ. , 58(1), 21–29. doi:10.1016/j.compedu.2011.07.007

Desai, K., Belmonte, U. H. H., Jin, R., Prabhakaran, B., Diehl, P., Ramirez, V. A., et al. (2017). “Experiences with multi-modal collaborative virtual laboratory (MMCVL),” in Proceedings - 2017 IEEE 3rd International Conference on Multimedia Big Data, BigMM , Laguna Hills, CA, USA , 19-21 April 2017 , 376–383. doi:10.1109/BigMM.2017.62

Dich, Y., Reilly, J., and Schneider, B. (2018). “Using physiological synchrony as an indicator of collaboration quality, task performance and learning,” in Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Midtown Manhattan, New York City: Springer International Publishing ). doi:10.1007/978-3-319-93843-1_8

Dixon, S. (2006). A history of virtual reality in performance. Int. J. Perform. Arts Digital Media 2 (1), 23–54. doi:10.1386/padm.2.1.23/1

Economou, D., Mitchell, W., Pettifer, S., Cook, J., and Marsh, J. (2001). “User centred virtual actor technology,” in Proceedings VAST 2001 virtual reality, archeology, and cultural heritage (New York, NY: Association for Computing Machinery ), 323–331. doi:10.1145/584993.585052

Edirisingha, P., Nie, M., Pluciennik, M., and Young, R. (2009). Socialisation for learning at a distance in a 3-D multi-user virtual environment. Br. J. Educ. Technol. 40 (3), 458–479. doi:10.1111/j.1467-8535.2009.00962.x

Franco, J. F., Da Cruz, S. R., and Lopes, R. D. D. (2006). “Computer graphics, interactive technologies and collaborative learning synergy supporting individuals’ skills development,” in ACM SIGGRAPH 2006 educators program, SIGGRAPH ’06 (New York, NY: Association for Computing Machinery ). doi:10.1145/1179295.1179338

Franco, J. F., and De Deus Lopes, R. (2009). “Three-dimensional digital enviroments and computer graphics influencing K-12 individuals’ digital literacy development and interdisciplinary lifelong learning,” in ACM SIGGRAPH ASIA 2009 educators program, SIGGRAPH ASIA ’09 (New York, NY: Association for Computing Machinery ). doi:10.1145/1666611.1666626

Freina, L., and Ott, M. (2015). “A literature review on immersive virtual reality in education: State of the art and perspectives,” in Proceedings of ELearning and Software for Education (ELSE) , Bucharest, Romania , April 23--24, 2015 .

Gerhard, M., Moore, D. J., and Hobbs, D. J. (2001). Continuous presence in collaborative virtual environments: Towards a hybrid avatar-agent model for user representation. Lect. Notes Comput. Sci. 2190, 137–155. doi:10.1007/3-540-44812-8_12

Girvan, C., and Savage, T. (2010). Identifying an appropriate pedagogy for virtual worlds: A communal constructivism case study. Comput. Educ. 55 (1), 342–349. doi:10.1016/j.compedu.2010.01.020

Girvan, C. (2018). What is a virtual world? Definition and classification. Educ. Technol. Res. Dev. 66 (5), 1087–1100. doi:10.1007/s11423-018-9577-y

Godin, J. J., and Pridmore, J. (2019). Using design thinking strategies and virtual reality in global virtual teams. Issues Inf. Syst. 20 (4), 104–106. doi:10.48009/4_iis_2019_104-106

Griol, D., Molina, J. M., and Callejas, Z. (2014). An approach to develop intelligent learning environments by means of immersive virtual worlds. J. Ambient Intell. Smart Environ. 6 (2), 237–255. doi:10.3233/AIS-140255

Herrera, F., Bailenson, J., Weisz, E., Ogle, E., and Zak, J. (2018). Building long-term empathy: A large-scale comparison of traditional and virtual reality perspective-taking. PLoS ONE 13, e0204494. doi:10.1371/journal.pone.0204494

PubMed Abstract | CrossRef Full Text | Google Scholar

Hwang, W. Y., and Hu, S. S. (2013). Analysis of peer learning behaviors using multiple representations in virtual reality and their impacts on geometry problem solving. Comput. Educ. 62, 308–319. doi:10.1016/j.compedu.2012.10.005

Jackson, R. L., Taylor, W., and Winn, W. (1999). “Peer collaboration and virtual environments: A preliminary investigation of multi-participant virtual reality applied in science education,” in Proceedings of the ACM symposium on applied computing (New York, NY: Association for Computing Machinery ), 121–125.

Jarmon, L., Traphagan, T., and Mayrath, M. (2008). Understanding project-based learning in Second Life with a pedagogy, training, and assessment trio. Educ. Media Int. 45 (3), 157–176. doi:10.1080/09523980802283889

Jensen, L., and Konradsen, F. (2018). A review of the use of virtual reality head-mounted displays in education and training. Educ. Inf. Technol. 23 (4), 1515–1529. doi:10.1007/s10639-017-9676-0

Jeong, H., and Hmelo-Silver, C. E. (2016). Seven affordances of computer-supported collaborative learning: How to support collaborative learning? How can technologies help? Educ. Psychol. 51 (2), 247–265. doi:10.1080/00461520.2016.1158654

Jeong, J. Y., Seo, S. J., Han, Y. J., Kim, S. J., and Jeong, H. (2018). “Learning English conversations in an immersive virtual reality environment,” in Icce 2018 - 26th international conference on computers in education, workshop proceedings (Philippines: Asia-Pacific Society for Computers in Education ), 638–643.

Jing, Y., Behzad, M., Jeff, H. P., Sarah Farukhi, A., S Khizer, K., Hinton, R. J., et al. (2018). Vital roles of β-catenin in trans-differentiation of chondrocytes to bone cells. Int. J. Ophthalmol. Clin. Res. 5 (2), 1–9. doi:10.7150/ijbs.23165

Joyner, D. A., Mavilakandy, A., Mittal, I., Kutnick, D., and Macintyre, B. (2021). Content - neutral immersive environments for cultivating scalable camaraderie . New York, NY: Association for Computing Machinery , 47–58.

Juliano, J. M., Spicer, R. P., Vourvopoulos, A., Lefebvre, S., Jann, K., Ard, T., et al. (2020). Embodiment is related to better performance on a brain–computer interface in immersive virtual reality: A pilot study. Sensors Switz. 20 (4), 1204. doi:10.3390/s20041204

Ke, F., and Lee, S. (2016). Virtual reality based collaborative design by children with high-functioning autism: Design-based flexibility, identity, and norm construction. Interact. Learn. Environ. 24 (7), 1511–1533. doi:10.1080/10494820.2015.1040421

Kleanthous, S., Michael, M., Samaras, G., and Christodoulou, E. (2016). “Transactive memory in task-driven 3d virtual world teams,” in ACM international conference proceeding series (New York, NY: Association for Computing Machinery ), 23–27. doi:10.1145/2971485.2996728

Kong, J. S. L., and Kwok, R. C. W. (2013). “Knowledge conversion in massive peers: A preliminary study on mutualistic co-presence,” in Proceedings - pacific asia conference on information systems, PACIS 2013 (Korea: Pacific Asia Conference on Information Systems (PACIS) ). https://www.scopus.com/inward/record.uri?eid=2-s2.0-84928485362&partnerID=40&md5=9a1b86ea4622e938ec393f4b968a7fc6 .

Kwon, C. (2019). Verification of the possibility and effectiveness of experiential learning using HMD-based immersive VR technologies. Virtual Real. 23 (1), 101–118. doi:10.1007/s10055-018-0364-1

Lattemann, C., and Stieglitz, S. (2012). “Challenges for lecturers in virtual worlds,” in Ecis 2012 - proceedings of the 20th European conference on information systems (Barcelona, Spain: European Conference on Information Systems (ECIS) ). https://www.scopus.com/inward/record.uri?eid=2-s2.0-84905717967&partnerID=40&md5=3fda0e424f60827726a346d39465e313 .

Le, Q. T., Pedro, A., and Park, C. S. (2015). A social virtual reality based construction safety education system for experiential learning. J. Intelligent Robotic Syst. Theory Appl. 79 (3–4), 487–506. doi:10.1007/s10846-014-0112-z

Li, H., Chan, G., and Skitmore, M. (2012). Multiuser virtual safety training system for tower crane dismantlement. J. Comput. Civ. Eng. 26 (5), 638–647. doi:10.1061/(asce)cp.1943-5487.0000170(asce)cp.1943-5487.0000170

Liu, Y., Wang, T., Wang, K., and Zhang, Y. (2021). Collaborative learning quality classification through physiological synchrony recorded by wearable biosensors. Front. Psychol. 12, 674369. doi:10.3389/fpsyg.2021.674369

Liu, Y. (2020). The application of virtual reality in empathy establishment: Foresee the future. Proc. Int. Conf. Comput. Intell. Appl. ICCIA 2020, 188–193. doi:10.1109/ICCIA49625.2020.00043

López-Faican, L., and Jaen, J. (2020). EmoFindAR: Evaluation of a mobile multiplayer augmented reality game for primary school children. Comput. Educ. 149, 103814. doi:10.1016/j.compedu.2020.103814

Makransky, G., and Lilleholt, L. (2018). A structural equation modeling investigation of the emotional value of immersive virtual reality in education. Educ. Technol. Res. Dev. 66 (5), 1141–1164. doi:10.1007/s11423-018-9581-2

Makransky, G., Terkildsen, T. S., and Mayer, R. E. (2019). Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learn. Instr. 60, 225–236. doi:10.1016/j.learninstruc.2017.12.007

McArdle, G., and Bertolotto, M. (2012). Assessing the application of three-dimensional collaborative technologies within an e-learning environment. Interact. Learn. Environ. 20 (1), 57–75. doi:10.1080/10494821003714749

McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochem. Medica 22 (3), 276–282. https://hrcak.srce.hr/89395 . doi:10.11613/bm.2012.031

Mehrfard, A., Fotouhi, J., Taylor, G., Forster, T., Navab, N., and Fuerst, B. (2019). A comparative analysis of virtual reality head-mounted display systems . arXiv . http://arxiv.org/abs/1912.02913 .

Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., and Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Comput. Educ. 70, 29–40. doi:10.1016/j.compedu.2013.07.033

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and Group, T. P. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 6 (7), e1000097. doi:10.1371/journal.pmed.1000097

Molka-Danielsen, J., and Brask, O. D. (2014). Designing virtual collaboration environment for distance learners: Exploring socialization as a basis for collaboration. Lect. Notes Bus. Inf. Process. 186, 74–89. doi:10.1007/978-3-319-09546-2_6

Moll, J., and Pysander, E. L. S. (2013). A haptic tool for group work on geometrical concepts engaging blind and sighted pupils. ACM Trans. Accessible Comput. 4 (4), 1–37. doi:10.1145/2493171.2493172

Moore, A. G., McMahan, R. P., and Ruozzi, N. (2021). Exploration of feature representations for predicting learning and retention outcomes in a VR training scenario. Big Data Cognitive Comput. 5 (3), 29. doi:10.3390/bdcc5030029

Mørch, A. I., Mifsud, L., and Eie, S. (2019). Developing a model of collaborative learning with minecraft for social studies classrooms using role-play theory and practice. Computer-Supported Collab. Learn. Conf. CSCL 1, 272–279.

Motejlek, J., and Alpay, E. (2019). “A taxonomy for virtual and augmented reality in education,” in Proceedings of the 46th SEFI annual conference 2018: Creativity, innovation and entrepreneurship for engineering education excellence ( arXiv ), 1089–1100.

Muhammad Nur Affendy, N., and Ajune Wanis, I. (2019). “A review on collaborative learning environment across virtual and augmented reality technology,” in IOP Conference Series: Materials Science and Engineering 5(1). doi:10.1088/1757-899X/551/1/012050

Mystakidis, S., Berki, E., and Valtanen, J. (2017). Toward successfully integrating mini learning games into social virtual reality environments – recommendations for improving open and distance learning. Edulearn17 Proc. 1, 968–977. doi:10.21125/edulearn.2017.1203

Nadolny, L., Woolfrey, J., Pierlott, M., and Kahn, S. (2013). SciEthics Interactive: Science and ethics learning in a virtual environment. Educ. Technol. Res. Dev. 61 (6), 979–999. doi:10.1007/s11423-013-9319-0

Nikolic, S., and Nicholls, B. (2018). Exploring student interest of online peer assisted learning using mixed-reality technology . Cham, Switzerland: Springer , 715, 48–54. doi:10.1007/978-3-319-73210-7_6

Nisiotis, L., and Kleanthous, S. (2019). “The relationship between students’ engagement and the development of transactive memory systems in MuvE: An experience report,” in Annual conference on innovation and technology in computer science education, ITiCSE, 71–77 (New York, NY: Association for Computing Machinery ). doi:10.1145/3304221.3319743

Nuñez, M., Quirós, R., Nuñez, I., Carda, J. B., and Camahort, E. (2008). Collaborative augmented reality for inorganic chemistry education. WSEAS Int. Conf. Proc. Math. Comput. Sci. Eng. 5, 271–277.

Olmos, E., Cavalcanti, J. F., Soler, J.-L., Contero, M., and Alcañiz, M. (2018). Mobile virtual reality: A promising technology to change the way we learn and teach (Singapore: Springer ), 95–106. doi:10.1007/978-981-10-6144-8_6

Parong, J., and Mayer, R. E. (2021). Cognitive and affective processes for learning science in immersive virtual reality. J. Comput. Assisted Learn. 37 (1), 226–241. doi:10.1111/jcal.12482

Pastel, S., Chen, C. H., Petri, K., and Witte, K. (2020). Effects of body visualization on performance in head-mounted display virtual reality. PLoS ONE 15, 0239226. doi:10.1371/journal.pone.0239226

Pellas, N. (2014). Harnessing and evaluating open sim for the implementation of an Inquiry-Based Collaborative Learning (Ib[C]L) script in computer science: Preliminary findings from a case study in higher education. Open Source Technol. Concepts, Methodol. Tools, Appl. 3–4, 1223–1246. doi:10.4018/978-1-4666-7230-7.ch059

Pellas, N., Peroutseas, E., and Kazanidis, I. (2013). “Virtual communities of inquiry (VCoI) for learning basic algorithmic structures with open simulator and Scratch4OS: A case study from the secondary education in Greece,” in ACM international conference proceeding series (New York, NY: Association for Computing Machinery ), 187–194. doi:10.1145/2490257.2490278

Radianti, J., Majchrzak, T. A., Fromm, J., and Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Comput. Educ. 147, 103778. doi:10.1016/j.compedu.2019.103778

Rebelo, F., Noriega, P., Duarte, E., and Soares, M. (2012). Using virtual reality to assess user experience. Hum. Factors 54 (6), 964–982. doi:10.1177/0018720812465006

Rogers, L. (2011). Developing simulations in multi-user virtual environments to enhance healthcare education. Br. J. Educ. Technol. 42 (4), 608–615. doi:10.1111/j.1467-8535.2010.01057.x

Rueda, J., and Lara, F. (2020). Virtual reality and empathy enhancement: Ethical aspects. Front. Robotics AI 7, 506984. doi:10.3389/frobt.2020.506984

Saldana, D., Neureither, M., Schmiesing, A., Jahng, E., Kysh, L., Roll, S. C., et al. (2020). Applications of head-mounted displays for virtual reality in adult physical rehabilitation: A scoping review. Am. J. Occup. Ther. 74 (5), 7405205060p1–7405205060p15. doi:10.5014/ajot.2020.041442

Scavarelli, A., Arya, A., and Teather, R. J. (2021). Virtual reality and augmented reality in social learning spaces: A literature review. Virtual Real. 25 (1), 257–277. doi:10.1007/s10055-020-00444-8

Schreiber, J., and Asnerly-Self, K. (2011). Educational research. The interrelationship of questions, sampling, design and analysis . Hoboken, New Jersey: John Wiley & Sons, Inc .

Shawky, D., Badawi, A., Said, T., and Hozayin, R. (2014). “Affordances of computer-supported collaborative learning platforms: A systematic review,” in Proceedings of 2014 international conference on interactive collaborative learning, ICL (New York, NY: Institute of Electrical and Electronics Engineers (IEEE) ), 633–651. doi:10.1109/ICL.2014.7017846

Shi, Y., Dua, J., Ahn, C. R., and Ragan, E. (2019). Impact assessment of reinforced learning methods on construction workers’ fall risk behavior using virtual reality. Automation Constr. 104, 197–214. doi:10.1016/j.autcon.2019.04.015

Sims, R., Chang, B., Bennett, V., Krishnan, A., Aboubakar, A., Coman, G., et al. (2022). “Step into my mind palace: Exploration of a collaborative paragogy tool in VR,” in Proceedings of 2022 8th international conference of the immersive learning research network, iLRN 2022. Immersive learning research network (New York, NY: Institute of Electrical and Electronics Engineers (IEEE) ). doi:10.23919/iLRN55037.2022.9815936

Smith, S. A. (2019). Virtual reality in episodic memory research: A review. Psychonomic Bull. Rev. 26 (4), 1213–1237. doi:10.3758/s13423-019-01605-w

Sousa Santos, B., Dias, P., Pimentel, A., Baggerman, J. W., Ferreira, C., Silva, S., et al. (2009). Head-mounted display versus desktop for 3D navigation in virtual reality: A user study. Multimedia Tools Appl. 41 (1), 161–181. doi:10.1007/s11042-008-0223-2

Srivastava, P., Rimzhim, A., Vijay, P., Singh, S., and Chandra, S. (2019). Desktop VR is better than non-ambulatory HMD VR for spatial learning. Front. Robotics AI 6, 50–15. doi:10.3389/frobt.2019.00050

Stahl, G., Koschmann, T., and Suthers, D. (2006). “Computer-supported collaborative learning: An historical perspective,” in Cambridge handbook of the learning sciences . Editor R. K. Sawyer (Cambridge: Cambridge University Press ), 409–426. http://gerrystahl.net/cscl/CSCL_Chinese_traditional.pdf .

Sulbaran, T., and Jones, L. F. (2012). “Utilizing a collaborative virtual reality environment as a training tool for construction students,” in ASEE annual conference and exposition, conference proceedings (Texas, TX: American Society for Engineering Education (ASEE) ). doi:10.18260/1-2--22208

Tan, C. S. S., Luyten, K., van Den Bergh, J., Schoning, J., and Coninx, K. (2014). The role of physiological cues during remote collaboration. Presence Teleoperators Virtual Environ. 23 (1), 90–107. doi:10.1162/PRES_a_00168

Terzidou, T., Tsiatsos, T., Dae, A., Samaras, O., and Chasanidou, A. (2012). “Utilizing virtual worlds for game based learning: Grafica, a 3D educational game in second life,” in Proceedings of the 12th IEEE international conference on advanced learning technologies, ICALT (New York, NY: Institute of Electrical and Electronics Engineers (IEEE) ), 624–628. doi:10.1109/ICALT.2012.55

Tovar, D. F., Jonker, V., and Hürst, W. (2020). Virtual reality and augmented reality in education: A review . Utrecht, Netherlands: Utrecht University .

Tremmel, C., Herff, C., Sato, T., Rechowicz, K., Yamani, Y., and Krusienski, D. J. (2019). Estimating cognitive workload in an interactive virtual reality environment using EEG. Front. Hum. Neurosci. 13, 401–412. doi:10.3389/fnhum.2019.00401

Tüzün, H., Bilgiç, H. G., and Elçi, S. Y. (2019). The effects of 3D multi-user virtual environments on collaborative learning and social presence. Int. Electron. J. Elem. Educ. 11 (3), 221–231. doi:10.26822/iejee.2019349247

Vallance, M., Ibayashi, K., and Goto, Y. (2015). Engineering active learning in 3D virtual worlds. Commun. Comput. Inf. Sci. 533, 268–282. doi:10.1007/978-3-319-22629-3_22

Vrellis, I., Papachristos, N. M., Bellou, J., Avouris, N., and Mikropoulos, T. A. (2010). Designing a collaborative learning activity in second life: An exploratory study in physics. Proc. - 10th IEEE Int. Conf. Adv. Learn. Technol. ICALT 2010, 210–214. doi:10.1109/ICALT.2010.65

Wang, C. J., Low, A. S. L., and Woo, C. S. (2014). “3D system for learning gains in the lower secondary normal (technical) stream,” in 3D Immersive and Interactive Learning (Singapore: Springer ), 93–103. doi:10.1007/978-981-4021-90-6_7

Wang, A., Thompson, M., Roy, D., Pan, K., Perry, J., Tan, P., et al. (2022). Iterative user and expert feedback in the design of an educational virtual reality biology game. Interact. Learn. Environ. 30 (4), 677–694. doi:10.1080/10494820.2019.1678489

Yeh, S. C., Hwang, W. Y., Wang, J. L., and Chen, Y. R. (2012). “Effects of multi-symbols on enhancing virtual reality based collaborative task,” in Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Berlin, Germany: Springer Berlin Heidelberg ), Vol. 7220, 101–111. doi:10.1007/978-3-642-31439-1_10

Yoon, H. J., Kim, J., Park, S. W., and Heo, H. (2020). Influence of virtual reality on visual parameters: Immersive versus non-immersive mode. BMC Ophthalmol. 20 (1), 200–208. doi:10.1186/s12886-020-01471-4

Zheng, L., Xie, T., and Liu, G. (2019). “Affordances of virtual reality for collaborative learning,” in Proceedings - international joint conference on information, media and engineering, ICIME 2018 (New York, NY: Institute of Electrical and Electronics Engineers (IEEE) ), 6–10. doi:10.1109/ICIME.2018.00011

Keywords: virtual reality, collaborative learning, virtual reality education, collaborative virtual environment, virtual reality and collaborative learning, collaborative virtual reality, collaborative virtual reality systems, educational technologies

Citation: van der Meer N, van der Werf V, Brinkman W-P and Specht M (2023) Virtual reality and collaborative learning: a systematic literature review. Front. Virtual Real. 4:1159905. doi: 10.3389/frvir.2023.1159905

Received: 06 February 2023; Accepted: 02 May 2023; Published: 19 May 2023.

Reviewed by:

Copyright © 2023 van der Meer, van der Werf, Brinkman and Specht. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Nesse van der Meer, [email protected]

• Conferences
• New Conferences
• search search
• You are not signed in

External Links

• Google Scholar
• HatzipanayiotiP19
• References: 0
• Cited by: 0
• Bibliographies: 0
• [Upload PDF for personal use]

Researchr is a web site for finding, collecting, sharing, and reviewing scientific publications, for researchers by researchers.

Sign up for an account to create a profile with publication list, tag and review your related work, and share bibliographies with your co-authors.

Collaborative Problem Solving in Local and Remote VR Situations

Adamantini Hatzipanayioti , Anastasia Pavlidou , Manuel Dixken , Heinrich H. Bülthoff , Tobias Meilinger , Matthias Bues , Betty J. Mohler . Collaborative Problem Solving in Local and Remote VR Situations . In IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019, Osaka, Japan, March 23-27, 2019 . pages 964-965 , IEEE, 2019. [doi]

• Bibliographies

Abstract is missing.

• Web Service API

UC Santa Cruz

Supporting Collaboration in Social VR

• Fernandes, Sean
• Advisor(s): Isbister, Katherine

Playful interactions have the potential to resolve some of the shortcomings inherent in collaboration in VR space, and offer a potential path for improving collaboration by taking greater advantage of the wide range of interactions possible in the virtual environment. Building toward this goal, this thesis experimentally evaluates two potential approaches to developing social VR collaborative experiences. The first approach offers a visual abstraction of conversational state, allowing participants to see the recent history of their conversation and adjust accordingly. The other introduces an ambient agent which does not participate in the conversation or provide any prescriptive advice, but instead passively indicates whether the conversations has become "imbalanced". They were evaluated through user tests involving a collaborative problem solving task in order to establish their effectiveness in social VR situations.

Enter the password to open this PDF file:

Collaborative Immersive Planning and Training Scenarios in VR

• Conference paper
• First Online: 19 May 2018
• Cite this conference paper

• Christian Eller 15 ,
• Timo Bittner 15 ,
• Marcus Dombois 15 &
• Uwe Rüppel 15  

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10863))

Included in the following conference series:

• Workshop of the European Group for Intelligent Computing in Engineering

3182 Accesses

7 Citations

Virtual reality is a promising technology to create fully immersive exercises in various fields such as civil safety engineering, security, teaching or facility management, designed as serious games for engineering purposes. Based on previous research and cooperation with industry partners and public institutions a concept for a collaborative virtual environment for multiple organizations with different hardware systems was devised. The approach focused on the integration of 1 st person virtual reality together with the corresponding interactions as well as a roles and rights management for all participants involved. Additionally, the setup allows for traditional desktop solutions to be included and enables users to build a dynamic and ever-changing training scenario based on an interactive planning phase. The concept was implemented and tested within a specific civil safety scenario using modern head-mounted displays as well as controllers and object tracking capabilities. This paper outlines the necessary steps in order to achieve such a collaborative virtual environment including various responsibilities for different roles and hierarchy levels and examines its potential. Together with the use of different hardware systems the described approach results in a flexible concept for planning and training applications in order to create dynamic, individual scenarios that facilitate the effectiveness of learning.

• Virtual reality
• Civil safety engineering
• Collaborative virtual environment
• Scenario planning and training
• Serious games

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Redaktion: Südbrandenburger Rettungskräfte proben Ernstfall in Cottbus. http://www.niederlausitz-aktuell.de/cottbus/68807/suedbrandenburger-rettungskraefte-proben-ernstfall-in-cottbus.html

Dörner, R., Broll, W., Grimm, P.F., Jung, B. (eds.): Virtual und Augmented Reality (VR/AR): Grundlagen und Methoden der Virtuellen und Augmentierten Realität. Springer, Heidelberg (2013)

Google Scholar  

Guillén-Nieto, V., Aleson-Carbonell, M.: Serious games and learning effectiveness: the case of it’s a deal! Comput. Educ. 58 , 435–448 (2012)

Article   Google Scholar  

Merchant, Z., Goetz, E.T., Cifuentes, L., Keeney-Kennicutt, W., Davis, T.J.: Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: a meta-analysis. Comput. Educ. 70 , 29–40 (2014)

Monahan, T., McArdle, G., Bertolotto, M.: Virtual reality for collaborative e-learning. Comput. Educ. 50 , 1339–1353 (2008)

Greenwald, S.W., Kulik, A., Kunert, A., Beck, S., Fröhlich, B., Cobb, S., Parsons, S., Newbutt, N., Gouveia, C., Cook, C., Snyder, A., Payne, S., Holland, J., Buessing, S., Fields, G., Corning, W., Lee, V., Xia, L., Maes, P.: Technology and Applications for Collaborative Learning in Virtual Reality (2017)

Lehner, V.D., DeFanti, T.A.: Distributed virtual reality: supporting remote collaboration in vehicle design. IEEE Comput. Graph. Appl. 17 , 13–17 (1997)

Wu, T.-H., Wu, F., Liang, C.-J., Li, Y.-F., Tseng, C.-M., Kang, S.-C.: A virtual reality tool for training in global engineering collaboration. Univ. Access Inf. Soc. 1–13 (2017). https://doi.org/10.1007/s10209-017-0594-0

Freeman, K.M., Thompson, S.F., Allely, E.B., Sobel, A.L., Stansfield, S.A., Pugh, W.M.: A virtual reality patient simulation system for teaching emergency response skills to U.S. navy medical providers. Prehosp. Disaster Med. 16 , 3–8 (2001)

Pucher, P.H., Batrick, N., Taylor, D., Chaudery, M., Cohen, D., Darzi, A.: Virtual-world hospital simulation for real-world disaster response: design and validation of a virtual reality simulator for mass casualty incident management. J. Trauma Acute Care Surg. 77 , 315 (2014)

Sharma, S., Jerripothula, S., Mackey, S., Soumare, O.: Immersive virtual reality environment of a subway evacuation on a cloud for disaster preparedness and response training. In: 2014 IEEE Symposium on Computational Intelligence for Human-Like Intelligence (CIHLI), pp. 1–6 (2014)

Passos, C., da Silva, M.H., Mol, A.C.A., Carvalho, P.V.R.: Design of a collaborative virtual environment for training security agents in big events. Cogn. Technol. Work 19 , 315–328 (2017)

Auggmed - The serious game platform. http://www.auggmed-project.eu/

XVR Simulation: Virtual Reality training software for safety and security. http://www.xvrsim.com/?t=gb

LUDUS - Virtual Reality for training in industry and emergencies. http://www.ludus-vr.com/en/

Regener, H., Hackstein, A.: Planspiel als Methode und Medium im taktik-und Führungstraining. Rettungsdienst 35 , 24 (2012)

Doeweling, S., Tahiri, T., Sowinski, P., Schmidt, B., Khalilbeigi, M.: Support for collaborative situation analysis and planning in crisis management teams using interactive tabletops. In: Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces, pp. 273–282. ACM, New York (2013)

Patig, S., Dibbern, J.: Requirements Engineering. http://www.enzyklopaedie-der-wirtschaftsinformatik.de/lexikon/is-management/Systementwicklung/Hauptaktivitaten-der-Systementwicklung/Problemanalyse-/Requirements-Engineering/index.html

Seyama, J., Nagayama, R.S.: The uncanny valley: effect of realism on the impression of artificial human faces. Presence: Teleoper. Virtual Environ. 16 , 337–351 (2007)

HTC Corporation: VIVE TM | Vive Tracker. https://www.vive.com/eu/vive-tracker/

Unity Technologies: Unity. https://unity3d.com

Exit Games: Multiplayer Game Development Made Easy | Photon Engine. https://www.photonengine.com/en-us/Photon

Number of Players Supported by Photon Cloud? — Photon Engine. https://forum.photonengine.com/discussion/3351/number-of-players-supported-by-photon-cloud

Welcome to VRTK · VRTK - Virtual Reality Toolkit. https://vrtoolkit.readme.io/docs

Cyberith GmbH: Virtual Reality Locomotion Virtualizer Cyberith. https://www.cyberith.com/

Download references

Author information

Authors and affiliations.

Technische Universität Darmstadt, Franziska-Braun-Str. 7, 64287, Darmstadt, Germany

Christian Eller, Timo Bittner, Marcus Dombois & Uwe Rüppel

You can also search for this author in PubMed   Google Scholar

Corresponding authors

Correspondence to Christian Eller , Timo Bittner , Marcus Dombois or Uwe Rüppel .

Editor information

Editors and affiliations.

Applied Computing and Mechanics Laboratory (IMAC), School of Architecture, Civil and Environmental Engineering (ENAC), Swiss Federal Institute of Technology, Lausanne (EPFL), Lausanne, Switzerland

Ian F. C. Smith

Institute for Landscape, Architecture, Construction and Territory (inPact) Construction and Environment Department (CED), University of Applied Sciences, Geneva (HEPIA), Geneva, Switzerland

Bernd Domer

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Cite this paper.

Eller, C., Bittner, T., Dombois, M., Rüppel, U. (2018). Collaborative Immersive Planning and Training Scenarios in VR. In: Smith, I., Domer, B. (eds) Advanced Computing Strategies for Engineering. EG-ICE 2018. Lecture Notes in Computer Science(), vol 10863. Springer, Cham. https://doi.org/10.1007/978-3-319-91635-4_9

Download citation

DOI : https://doi.org/10.1007/978-3-319-91635-4_9

Published : 19 May 2018

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-91634-7

Online ISBN : 978-3-319-91635-4

eBook Packages : Computer Science Computer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research