• Open access
  • Published: 15 December 2023

Preliminary guideline for reporting bibliometric reviews of the biomedical literature (BIBLIO): a minimum requirements

  • Ali Montazeri   ORCID: orcid.org/0000-0002-5198-9539 1 ,
  • Samira Mohammadi 1 ,
  • Parisa M.Hesari 2 ,
  • Marjan Ghaemi 3 ,
  • Hedyeh Riazi 4 &
  • Zahra Sheikhi-Mobarakeh 5  

Systematic Reviews volume  12 , Article number:  239 ( 2023 ) Cite this article

2083 Accesses

29 Altmetric

Metrics details

A bibliometric review of the biomedical literature could be essential in synthesizing evidence if thoroughly conducted and documented. Although very similar to review papers in nature, it slightly differs in synthesizing the data when it comes to providing a pile of evidence from different studies into a single document. This paper provides a preliminary guideline for reporting bibliometric reviews of the biomedical literature (BIBLIO).

The BIBLIO was developed through two major processes: literature review and the consensus process. The BIBLIO started with a comprehensive review of publications on the conduct and reporting of bibliometric studies. The databases searched included PubMed, Scopus, Web of Sciences, and Cochrane Library. The process followed the general recommendations of the EQUATOR Network on how to develop a reporting guideline, of which one fundamental part is a consensus process. A panel of experts was invited to identify additional items and was asked to choose preferred options or suggest another item that should be included in the checklist. Finally, the checklist was completed based on the comments and responses of the panel members in four rounds.

The BIBLIO includes 20 items as follows: title (2 items), abstract (1 item), introduction/background (2 items), methods (7 items), results (4 items), discussion (4 items). These should be described as a minimum requirements in reporting a bibliometric review.

Conclusions

The BIBLIO for the first time provides a preliminary guideline of its own kind. It is hoped that it could contribute to the transparent reporting of bibliometric reviews. The quality and utility of BIBILO remain to be investigated further.

Peer Review reports

Several guidelines exist for reporting findings of different study designs. The detailed explanations and checklists for such guidelines can be found in Enhancing the Quality and Transparency of Health Research (EQUATOR) Network [ 1 ] and are available to research communities [ 2 ]. For instance, the quality of reporting of meta-analyses (QUOROM) statement for improving the quality of reporting meta-analyses of randomized controlled trials was first published in The Lancet in 1999 [ 3 ]. Consequently, the work was further improved, and it was replaced with the preferred reporting items for systematic reviews and meta-analyses (PRISMA) [ 4 ]. This guideline was published simultaneously in 6 journals in 2009 [ 4 , 5 , 6 , 7 , 8 , 9 ], and since then, many biomedical journals and investigators have adhered to this instruction. The instruction also was extended, and complementary versions of the guideline either are developed (such as PRISMA for Abstracts) [ 10 ] or are under development (e.g., PRISMA for children) [ 11 ]. Even the preferred reporting items for overviews of reviews (PRIOR) are proposed [ 12 ], and a recent call by Systematic Reviews (the journal) indicates that attempts to enhance the knowledge of this type of reporting are in progress [ 12 , 13 ].

However, we believe there is also a need for a guideline for another type of reporting, namely, Guideline for Reporting Bibliometric Reviews of the Biomedical Literature (BIBLIO). A bibliometric or a bibliographic review of the literature is different from an overview. Recently, the literature witnessed a relatively considerable number of bibliometric analyses of the biomedical literature [ 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ]. The number of publications related to various topics with bibliometric or bibliography/bibliographic in the title during the last 10 years is presented in Fig.  1 . Therefore, this paper attempts to propose a preliminary version of a guideline for reporting bibliometric reviews of the literature. The guideline was developed based on all existing guidelines presented in the EQUATOR Network [ 1 ]. In addition, experiences from writing a number of bibliometric reviews [ 24 , 25 , 26 , 27 , 28 ] helped the authors to formulate this first version of the work with the courage that it could be improved further by receiving feedbacks from other scholars in the field.

figure 1

Papers with bibliography/bibliographic and bibliometric in the title of publications during 2013–2022 (PubMed)

Although BIBLIO is in its preliminary stage of development and there is no evidence of its quality and utility, it is hoped that it could contribute to the transparent reporting of bibliometric reviews. The application of bibliometric reviews enables one to analyze vast amounts of publications and their production patterns on macroscopic and microscopic levels [ 29 ]. Therefore, this study aimed to provide a guideline for reporting bibliometric reviews. The BIBLIO checklist was registered in the EQUATOR Network on 19 October 2021 [ 30 ].

The term bibliometric and bibliography are used interchangeably in the literature. Earlier, the term bibliography was more popular, but it was gradually replaced with the bibliometric expression (Fig.  2 ). The history of the statistical bibliography as reviewed by Thackray [ 31 ] indicates that the root goes back to early 1900s as this was acknowledged in a paper by Garfield [ 32 ] and a number of scholar such as Cole and Eames (1917), Hulme (1923), Lotka (1926), and Gross and Gross (1927) were listed as those who contributed to the technic of statistical analysis of the literature. However, it was Otlet in 1934 who first used the term “bibliometrie” and defined it as “the measurement of all aspects related to the publication and reading of books and documents” [ 33 , 34 ]. Then in 1969, Pritchard coined the term “bibliometrics” and defined it as “all the studies which seek to quantify the processes of written communication” [ 35 ]. The detailed history since 1934 is presented in Table  1 .

figure 2

Trends of using bibliography/bibliographic or bibliometric in the title of publications during 2013–2022 (PubMed)

Bibliometric is a type of review that can be used to look at different and important areas of investigations and obtain a general synopsis of published literature [ 39 ]. This guideline defines a bibliometric review as follows “a review of all full published papers that appear in the biomedical journals and includes all types of evidence such as descriptive studies, observational studies, experimental studies, qualitative studies, and systematic reviews in order to account for every single evidence exist. The bibliometric of the literature does not include electronic publications a head of print since the ultimate date for such publications are not known”. This definition was formulated based on chronological account of the term bibliometric and its developments [ 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 ].

Similarities and differences between systematic reviews and bibliometrics

Bibliometric is similar to systematic review in retrieving the literature [ 40 ], but they have low agreement rate regarding relevant literature and the purpose. While systematic reviews are seeking to respond to a very clear question based on good quality evidences, bibliometrics is rather a numeration of evidence without quality assessment. Bibliometrics often rely on the interpretation of quantitative details of publications such as main topics, authors, sources, most impactful authors, most impactful articles, and countries in a particular area in the existing literature. In this type of study, mapping techniques including graphical representations, tabulated forms, network diagrams, and so on are used to present results usually performing these with the assistance of softwares [ 39 , 40 , 41 , 42 ].

Development of BIBLIO

The BIBLIO was developed through two major processes: literature review and the consensus process. These are briefly described as follows:

1. Literature review for item selection

The BIBLIO started with a comprehensive review to identify potential items for including in this guideline. The databases searched included PubMed, Scopus, Web of Sciences, and Cochrane Library. The aim was to examine and review all methodological papers on the conduct and reporting of bibliometric studies up to 2021. The search was updated in January 2022 and once during the process of revisions in September 2023. Papers were retrieved using different keywords and MeSH terms including “bibliometric,” “bibliography,” and “bibliographic” in the title of papers. All potentially relevant publications were extracted and reviewed independently by two authors (AM and SM). Overall, 13,720 papers were identified. After removing duplicates and irrelevant documents, only 19 papers [ 40 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 ] were found that were dealing with methodological issues. Also, we visited all reporting guidelines for review studies that are indexed in the EQUATOR [ 1 ]. The items derived from the literature are shown in Table  2 .

2. Consensus process

The process followed the general recommendations of the EQUATOR Network on how to develop a reporting guideline, of which one fundamental part is a consensus process [ 1 ]. We used Delphi consensus to obtain advice on how to report a “bibliometrics.” Delphi was performed based on the conducting and reporting Delphi studies (CREDES) guideline [ 61 ]. A panel consisted of eleven experts, including bibliometrician, epidemiologist, clinician, librarian, statistician, journal editor, and a research fellow. They were invited to see the list of items derived from the previous stage and asked to identify additional items and to choose preferred options or suggest other items that should be included in the checklist. In each round of the Delphi, the feedback process allowed and encouraged the selected participants to review and assess their own initial judgments. Thus, the results of previous iterations regarding specific items were changed or modified by each member of the expert panel in later iterations based on the review and assessing the comments and feedback provided by the other Delphi panelists [ 62 ].

In the first round of the Delphi process, we used an open-ended questionnaire to solicit specific information and to add suggested items to the list of items and increase the rich of data collection. After receiving the experts’ responses, we converted the collected information into a well-structured questionnaire on a five-point scale with content analysis technique. This questionnaire was used as the survey questionnaire for the second round of data collection. Each Delphi participant received a second questionnaire and was asked to review the items summarized based on the information provided in the first round. Accordingly, we asked Delphi panelists to rate items and state the rationale concerning rating priorities. In the third round, each Delphi panelist received a questionnaire that included the items and ratings summarized in the previous round and was asked to revise their judgments. The remaining items, ratings, minority opinions, and items achieved consensus were distributed to the panelists in the final round. The fourth round provided a final opportunity for participants to revise their responses after formal feedback of the group. At last, the checklist was finalized based on the comments and answers of the panel members in four rounds. The cut-off for consensus was determined by percentage of agreement (mainly 75 to 80%). The duration of each round of Delphi was about 8 weeks, and the length of the overall study process was 8 months. Before beginning the Delphi survey, all experts were asked to disclose any conflicts of interest. The response rate was 100% for all four rounds of the Delphi process.

Scope of the guideline

BIBLIO is for use in reporting bibliometric reviews and has been designed primarily for bibliometric reviews that evaluate published papers irrespective of the design of the studies. The BIBLIO items are relevant for all types of quantitative and qualitative studies. BIBLIO can be used for reporting original bibliometric reviews and updated bibliometric reviews. BIBLIO is not to guide a bibliometric review conduct. However, familiarity with BIBLIO is helpful when planning and conducting bibliometric reviews to ensure that all recommended information is captured.

The BIBLIO checklist

The development team provided a list of items based on the literature review and presented them into the consensus process. Participants made revisions to the phrasing and format of the checklist by consolidating and eliminating items during the consensus process. Eventually, the BIBLIO checklist consisted of 20 items that should be described as a minimum requirements in reporting a bibliometric review as follows: title (2 items), abstract (1 item), introduction/background (2 items), methods (7 items), results (4 items), discussion (4 items). The full description of the items is in progress and will be available in due course. However, as an example here, we elaborate on item 15. As shown item 15 provides guidelines for reporting the results. As such four options are proposed. In the following section, we describe each option ensuring that examples given could help investigators to better summarize the findings. Since the opening part of each option is the same here the focus is on how organize the main findings:

Option 1: Organization based on study design and main study types

Research design is a blueprint of a scientific study. We could summarize studies based on different designs and main study types. For instance, one might summarize main study types based on randomized trials, observational studies, study protocols, diagnostic/prognostic studies, case reports, clinical practice guidelines, and qualitative studies on a given topic.

Option 2: Organization based on outcome measures

The other suggestive way to summarize the main findings is based on outcomes. For instance, a bibliometric analysis that evaluated the impact of race on postoperative outcomes and complications following elective spine surgery was classified based on outcomes providing four categories including general complications, medical complications, surgical complications, and postoperative outcomes [ 63 ].

Option 3: Organization based on concept

To simplify and clarify this presentation approach, we explain this option with an example. A study on bibliometric analysis of health literacy instruments summarized the findings in four tables according to the concept behind instruments including general instruments, condition-specific health literacy instruments (disease and content), population-specific instruments, and electronic health [ 28 ]. Authors could invent such concepts or use the literature for categorizing and summarizing the findings.

Option 4: Organization based on different subtitles relevant to the main topic

This presentation approach is well known and was used in many studies. One example for this option is a bibliometric study on health-related quality of life in breast cancer patients. In this study, the findings were summarized and presented according to treatments modalities and a number of classifications including surgical treatment, systemic therapies, psychological distress, supportive care, and common symptoms [ 26 ]. One should note there are many ways that we could summarize and tabulate the findings to provide a quick and at the same time a comprehensive perspectives of the studies under review. The checklist is presented in Table  3 .

A bibliometric review is a helpful means for accurately and reliably summarizing the evidence, specifically when a large number of papers exist on a given topic [ 69 ]. The bibliometric studies that are well done usually could help to grasp the current literature, identify knowledge gaps, derive novel ideas for investigation, and position their intended contributions to the field [ 43 ].

The bibliometric methods are quantitative and descriptive by nature but also used to make pronouncements about qualitative aspects. The principal purpose of bibliometric studies is to change intangible knowledge (scientific quality) into manageable entities [ 70 ]. Bibliometrics are not in-depth and evaluative reviews. However, they could briefly report on effectiveness and evaluations. Overall, a good bibliometric review should provide a take-home message for its readers.

A number of recommendations are proposed to improve readability of bibliometric reviews. For instance, it was proposed using easy-to-interpret metrics, as non-experts have a difficulty understanding of complex indicators. Also, it was recommended to avoid inventing the indicators, especially composite metrics that mix several indicators in a single measure. Likewise, it was suggested to avoid conscious efforts to manipulate the findings, for instance, choosing metrics that may favor your institution, certain areas, or researchers within it [ 44 ].

A bibliometric review could reveal how much effort has been made into a specific topic. In addition, presenting and summarizing the studies allows scholars to use bibliometric analysis to uncover emerging trends in article publishing, journals’ performances, collaboration patterns, and exploring the intellectual structure of a specific domain in the extant literature [ 71 , 72 ]. Describing the evidence could help policymakers, managers, and other decision-makers to formulate appropriate recommendations for practice or policy [ 73 ] and help editors judge the merits of publishing reports of new studies [ 74 ]. The bibliometric also helps translate and map the cumulative scientific knowledge and evolutionary nuances of well-established fields by making sense of large volumes of unstructured data in rigorous ways [ 43 ].

The use of BIBLIO similar to other guidelines [ 3 , 4 , 75 ] has the potential to benefit many stakeholders. The BIBLIO provides readers with a complete understanding of evidence about the necessity of each item. We have attempted to ensure that the guideline is helpful to authors seeking guidance on what to include in a bibliometric review. We hope the BIBLIO will help increase the quality of reported and published bibliometric reviews. Peer reviewers, editors, and other interested readers might also find the BIBLIO helpful in assessing such reviews. We hope journal editors will encourage authors to include the BIBLIO checklist when submitting a bibliometric review for publication.

Finally, although we followed the general recommendations of the EQUATOR Network and used a literature review and a Delphi consensus process to develop the BIBLIO checklist, it seems that its main limitation is the fact that there is no evidence to suggest it will improve the quality of bibliometric reviews. In this regard, feedback from editors and researchers about details and overall structure can be helpful. Additionally, one should note that bibliometric reviews is not an in-depth review of the literature and rather the most important contribution of this type of reviews is to collect and summarize evidence when we witness a pile of evidence on a topic. As such it reveals that how much effort has been conducted on a topic. In addition, this approach might help investigators to create new questions to conduct more focused studies on the topic in the future [ 26 ].

The BIBLIO provides a reporting guideline for bibliometric reviews of the biomedical literature. We hope that the guideline could result in more transparent and accurate reporting of bibliometric reviews.

Availability of data and materials

Not applicable.

Abbreviations

Enhancing the Quality and Transparency of Health Research

Preferred reporting items for systematic reviews and meta-analyses

Preferred reporting items for overviews of reviews

Guideline for reporting bibliometric review of the biomedical literature

Equator Network. Reporting guidelines for main study types. 2022. http://www.equator-network.org/ .

Google Scholar  

Groves T. Enhancing the quality and transparency of health research. BMJ. 2008;337(7661):a718.

Article   PubMed   Google Scholar  

Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Lancet. 1999;354(9193):1896–900.

Article   CAS   PubMed   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman D, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264–9.

Moher D, Liberati A, Tetzlaff J, Altman DG, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Open Med. 2009;3(3):e123–30.

PubMed   PubMed Central   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006–12.

Moher D, Liberati A, Tetzlaff J, Altman DG, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

Article   PubMed   PubMed Central   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg. 2010;8(5):336–41.

Moher D, Liberati A, Tetzlaff J, Altman DG, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Plos Med. 2009;6(7):e1000097.

Transparent reporting of systematic reviews and meta-analyses. 2021. http://prisma-statement.org/Extensions/ .

Transparent reporting of systematic reviews and meta-analyses. 2021. http://www.prisma-statement.org/Extensions/InDevelopment .

Pollock M, Fernandes RM, Becker LA, Featherstone R, Hartling L. What guidance is available for researchers conducting overviews of reviews of healthcare interventions? A scoping review and qualitative metasummary. Syst Rev. 2016;5(1):190.

Hunt H, Pollock A, Campbell P, Estcourt L, Brunton G. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev. 2018;7(1):39.

Al-Jabi SW. Arab world’s growing contribution to global leishmaniasis research (1998–2017): a bibliometric study. BMC Public Health. 2019;19(1):625.

Bullock N, Ellul T, Bennett A, Steggall M, Brown G. The 100 most influential manuscripts in andrology: a bibliometric analysis. Basic Clin Androl. 2018;28:15.

Huang YK, Hanneke R, Jones RM. Bibliometric analysis of cardiometabolic disorders studies involving NO2, PM2.5 and noise exposure. BMC Public Health. 2019;19(1):877.

Jarden RJ, Narayanan A, Sandham M, Siegert RJ, Koziol-McLain J. Bibliometric mapping of intensive care nurses’ wellbeing: development and application of the new iAnalysis model. BMC Nurs. 2019;18:21.

Lewis R, Hendry M, Din N, Stanciu MA, Nafees S, Hendry A, et al. Pragmatic methods for reviewing exceptionally large bodies of evidence: systematic mapping review and overview of systematic reviews using lung cancer survival as an exemplar. Syst Rev. 2019;8(1):171.

Mahavadi A, Shah AH, Sarkiss CA. Commentary: a bibliometric analysis of neurosurgical practice guidelines. Neurosurgery. 2020;86(5):E412–3.

Okoroiwu HU, Lopez-Munoz F, Povedano-Montero FJ. Bibliometric analysis of global Lassa fever research (1970–2017): a 47–year study. BMC Infect Dis. 2018;18(1):639.

Shi G, Liu N, Yu X, Zhang H, Li S, Wu S, et al. Bibliometric analysis of medical malpractice literature in legal medicine from 1975 to 2018: Web of Science review. J Forensic Leg Med. 2019;66:167–83.

Sweileh WM, Al-Jabi SW, Saed HZ, Sawalha AF. Outdoor air pollution and respiratory health: a bibliometric analysis of publications in peer-reviewed journals (1900–2017). Multidiscip Respir Med. 2018;13:15.

Gorraiz JI, Repiso R, De Bellis N, Deinzer G. Best practices in bibliometrics & bibliometric services. Front Res Metr Anal. 2021;6:771999.

Montazeri A, McEwen J, Gillis CR. Quality of life in patients with ovarian cancer: current state of research. Support Care Cancer. 1996;4(3):169–79.

Montazeri A, Gillis CR, McEwen J. Quality of life in patients with lung cancer: a review of literature from 1970 to 1995. Chest. 1998;113(2):467–81.

Montazeri A. Health-related quality of life in breast cancer patients: a bibliographic review of the literature from 1974 to 2007. J Exp Clin Cancer Res. 2008;27:32.

Montazeri A. Quality of life data as prognostic indicators of survival in cancer patients: an overview of the literature from 1982 to 2008. Health Qual Life Outcomes. 2009;7:102.

Tavousi M, Mohammadi S, Sadighi J, Zarei F, Kermani RM, Rostami R, Montazeri A. Measuring health literacy: a systematic review and bibliometric analysis of instruments from 1993 to 2021. PLoS One. 2022;17(7):e0271524.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Kokol P, Blazun Vosner H, Zavrsnik J. Application of bibliometrics in medicine: a historical bibliometrics analysis. Health Info Libr J. 2021;38(2):125–38.

Enhancing the Quality and Transparency of Health Research (EQUATOR). Reporting guidelines for main study types. 2021. https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-other-studydesigns/#BIBLIO .

Thackray A. Measurement in the historiography of science. In: Elkana YJ, Lederberg JK, K.Merton R, Thackray A, Zuckerman H, editors. Toward a metric of science: the advent of science indicator. New York: John Wiley & Sons; 1978. p. 11–30.

Garfield E. The intended consequences of Robert K. Merton Scientometrics. 2004;60(1):51–61.

Article   Google Scholar  

Otlet P. Library science: forgotten founder of bibliometrics. Nature. 1934;510:218.

Wikipedia. 2022. https://en.wikipedia.org/wiki/Bibliometrics .

Pritchard A. Statistical bibliography or bibliometrics. J Doc. 1969;25(4):348–9.

Hawkins DT. Bibliometrics of electronic journals in information science. Inf Res. 2001;7(1):7–11.

De Bellis N. Bibliometrics and citation analysis: from the science citation index to cybermetrics. Lanham–Toronto–Plymouth: Scarecrow Press; 2009.

Broadus RN. Toward a definition of ‘bibliometrics.’ Scientometrics. 1987;12:373–9.

Kumar M, George RJ, PS A. Bibliometric analysis for medical research. Indian J Psychol Med. 2023;45(3):277–82.

Lubowitz JH, Brand JC, Rossi MJ. Search methods for systematic reviews and bibliographic articles can improve: responsibilities of authorship are vast. Arthroscopy. 2023;39(6):1367–8.

Aria M, Cuccurullo C. Bibliometrix: an R-tool for comprehensive science mapping analysis. J Informetr. 2017;11(4):959–75.

Rojas-Sanchez MA, Palos-Sanchez PR, Folgado-Fernandez JA. Systematic literature review and bibliometric analysis on virtual reality and education. Educ Inf Technol. 2023;28(1):155–92.

Donthu N, Kumar S, Mukherjee D, Pandey N, Lim WM. How to conduct a bibliometric analysis: an overview and guidelines. J Bus Res. 2021;133:285–96.

Cabezas-Clavijo A, Torres-Salinas D. Bibliometric reports for institutions: best practices in a responsible metrics scenario. Front Res Metr Anal. 2021;6:696470.

Tester AC. A method for making a bibliography. Science. 1865;1930(72):321–2.

Ayala YD, Landero C. Practical method of determining the direction of the electrical axis in the frontal plane; bibliography of literature concerning methods of determining it. Arch Cardiol Mex. 1958;28(2):235–43.

Raisig LM. Mathematical evaluation of the scientific serial: improved bibliographic method offers new objectivity in selecting and abstracting the research journal. Science. 1960;131(3411):1417–9.

Bachev VI. Method of bibliographic work in performing local historical medical research. Sov Zdravookhr. 1979;1:61–2.

Ninkov A, Frank JR, Maggio LA. Bibliometrics: methods for studying academic publishing. Perspect Med Educ. 2022;11(3):173–6.

Kho ME, Brouwers MC. The systematic review and bibliometric network analysis (SeBriNA) is a new method to contextualize evidence. Part 1: description. J Clin Epidemiol. 2012;65(9):1010–5.

Campos-Asensio C. How to develop a bibliographic search strategy? Enferm Intensiva. 2018;29(4):182–6.

Article   CAS   Google Scholar  

Chiang HS, Huang RY, Weng PW, Mau LP, Tsai YC, Chung MP, et al. Prominence of scientific publications towards peri-implant complications in implantology: a bibliometric analysis using the H-classics method. J Oral Rehabil. 2018;45(3):240–9.

Van de Laar L, De Kruif T, Waltman L, Meijer I, Gupta A, Hagenaars N. Improving the evaluation of worldwide biomedical research output: classification method and standardised bibliometric indicators by disease. BMJ Open. 2018;8(6):e020818.

Linnenluecke MK, Marrone M, Singh AK. Conducting systematic literature reviews and bibliometric analyses. Aust J Manag. 2020;45(2):175–94.

Jappe A. Professional standards in bibliometric research evaluation? A meta-evaluation of European assessment practice 2005–2019. PLoS One. 2020;15(4):e0231735.

Schultz F, Anywar G, Quave CL, Garbe LA. A bibliographic assessment using the degrees of publication method: medicinal plants from the rural greater Mpigi region (Uganda). Evid Based Complement Alternat Med. 2021;2021:6661565.

Moro G, Valgimigli L. Efficient self-supervised metric information retrieval: a bibliography based method applied to COVID literature. Sensors. 2021;21(19):6430.

Fernández-Ananín S, Rodríguez JB, Soler EM. Then, how do I document the idea? Bibliographic search engines. Cir Esp. 2022;100(6):375–7.

Liu F. Retrieval strategy and possible explanations for the abnormal growth of research publications: re-evaluating a bibliometric analysis of climate change. Scientometrics. 2023;128(1):853–9.

Koo M, Lin SC. An analysis of reporting practices in the top 100 cited health and medicine-related bibliometric studies from 2019 to 2021 based on a proposed guidelines. Heliyon. 2023;9(6):e16780.

Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on conducting and reporting Delphi studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017;31(8):684–706.

Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Evaluation. 2007;12(10):2.

Akosman I, Kumar N, Mortenson R, Lans A, De La Garza Ramos R, Eleswarapu A, et al. Racial differences in perioperative complications, readmissions, and mortalities after elective spine surgery in the United States: a systematic review using AI-assisted bibliometric analysis. Glob Spine J. 2023:21925682231186759. Online ahead of print.

McDougal L, Dehingia N, Cheung WW, Dixit A, Raj A. COVID-19 burden, author affiliation and women’s well-being: a bibliometric analysis of COVID-19 related publications including focus on low-and middle-income countries. eClinicalMedicine. 2022;52:101606.

Henstock L, Wong R, Tsuchiya A, Spencer A. Behavioral theories that have influenced the way health state preferences are elicited and interpreted: a bibliometric mapping analysis of the time trade-off method with VOSviewer visualization. Front Health Serv. 2022;2:848087.

Bodea F, Bungau SG, Negru AP, Radu A, Tarce AG, Tit DM, et al. Exploring new therapeutic avenues for ophthalmic disorders: glaucoma-related molecular docking evaluation and bibliometric analysis for improved management of ocular diseases. Bioengineering. 2023;10(8):983.

Sang XZ, Wang CQ, Chen W, Rong H, Hou LJ. An exhaustive analysis of post-traumatic brain injury dementia using bibliometric methodologies. Front Neurol. 2023;14:1165059.

Ramli MI, Hamzaid NA, Engkasan JP, Usman J. Respiratory muscle training: a bibliometric analysis of 60 years’ multidisciplinary journey. Biomed Eng Online. 2023;22(1):50.

Szomszor M, Adams J, Fry R, Gebert C, Pendlebury DA, Potter RW, Rogers G. Interpreting bibliometric data. Front Res Metr Anal. 2021;5:628703.

Wallin JA. Bibliometric methods: pitfalls and possibilities. Basic Clin Pharmacol Toxicol. 2005;97(5):261–75.

Donthu N, Kumar S, Pandey N, Lim WM. Research constituents, intellectual structure, and collaboration patterns in journal of international marketing: an analytical retrospective. J Int Mark. 2021;29(2):1–25.

Verma S, Gustafsson A. Investigating the emerging COVID-19 research trends in the field of business and management: a bibliometric analysis approach. J Bus Res. 2020;118:253–61.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev. 2021;10:89.

Young C, Horton R. Putting clinical trials into context. Lancet. 2005;366(9480):107–8.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021;88:105906.

Download references

Acknowledgements

Author information, authors and affiliations.

Population Health Research Group, Health Metrics Research Center, Iranian Institute for Health Sciences Research, ACECR, Tehran, Iran

Ali Montazeri & Samira Mohammadi

Department of Epidemiology and Biostatistics, Schulich School of Medicine and Dentistry, Western University, London, Canada

Parisa M.Hesari

Vali-E-Asr Reproductive Health Research Center, Family Health Research Institute, Tehran University of Medical Sciences, Tehran, Iran

Marjan Ghaemi

Department of Midwifery and Reproductive Health, School of Nursing and Midwifery, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Hedyeh Riazi

Quality of Life Research Groups, Breast Cancer Research Center, Motamed Cancer Institute, ACECR, Tehran, Iran

Zahra Sheikhi-Mobarakeh

You can also search for this author in PubMed   Google Scholar

Contributions

AM put forward the idea, proposed the guideline, did the literature search, and wrote the manuscript. SM performed literature search, helped in writing the manuscript, and provided the figures and tables. PMH performed literature search and provided help in writing process. MGH helped in writing process. HR contributed to writing and conducting group discussion. ZSM helped in literature search and writing process. All authors read and approved the manuscript.

Corresponding author

Correspondence to Ali Montazeri .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no conflict of interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Montazeri, A., Mohammadi, S., M.Hesari, P. et al. Preliminary guideline for reporting bibliometric reviews of the biomedical literature (BIBLIO): a minimum requirements. Syst Rev 12 , 239 (2023). https://doi.org/10.1186/s13643-023-02410-2

Download citation

Received : 03 August 2022

Accepted : 04 December 2023

Published : 15 December 2023

DOI : https://doi.org/10.1186/s13643-023-02410-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Reporting guideline
  • Bibliometric reviews
  • Bibliography

Systematic Reviews

ISSN: 2046-4053

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

a literature review with bibliometric analysis

Advertisement

Advertisement

Sustainable development goals: a bibliometric analysis of literature reviews

  • Review Article
  • Published: 24 November 2022
  • Volume 30 , pages 5502–5515, ( 2023 )

Cite this article

  • Natália Ueda Yamaguchi   ORCID: orcid.org/0000-0002-8806-2454 1 ,
  • Eduarda Gameleira Bernardino 1 ,
  • Maria Eliana Camargo Ferreira 1 ,
  • Bruna Pietroski de Lima 1 ,
  • Mauro Renato Pascotini 2 &
  • Mirian Ueda Yamaguchi 2  

7495 Accesses

12 Citations

1 Altmetric

Explore all metrics

The research in sustainable development goals (SDG) increases year by year since its approval in 2015. Typically, after a phase of exponential growth, the number of publications increases at lower rates, suggesting a consolidation process in which literature reviews become a relevant and high-evidence type of document. In this context, the aim of this study was to perform an unprecedented bibliometric analysis of literature reviews on SDG to assess the evolution and consolidation of the scientific research. Article reviews on SDG from 2015 to 2022 were retrieved from Web of Science core collection and a descriptive bibliometric analysis was performed by growth rate, research area, source, citation, and region. Mapping and cluster analysis using keyword co-occurrence, co-authorship, and bibliographic coupling were also applied. The result revealed that SDG is a fast-growing field, with a trend in the diversification of research areas. Most of the review documents were categorized in general aspects of sustainability. Technology (SDG 9) and economic growth (SDG 8) were spotted as hidden key research areas. This result is contrary to previous bibliometric studies on SDG, demonstrating the rapid evolution and change in the field. In addition, literature reviews on reduced inequalities (SDG 10), gender equality (SDG 5); oceans, seas, and marine environments (SDG 14); and peace, justice, and strong institutions (SDG 16) were revealed as research gaps. Thus, the results demonstrated that the research on SDG cannot yet be considered a consolidated area of research, as it leaves many SDG unexplored. Future research has been proposed accordingly.

Similar content being viewed by others

a literature review with bibliometric analysis

Three pillars of sustainability: in search of conceptual origins

Ben Purvis, Yong Mao & Darren Robinson

a literature review with bibliometric analysis

Responsible Design Thinking for Sustainable Development: Critical Literature Review, New Conceptual Framework, and Research Agenda

Brian Baldassarre, Giulia Calabretta, … Erik Jan Hultink

a literature review with bibliometric analysis

Pathways from research to sustainable development: Insights from ten research projects in sustainability and resilience

Anna Scaini, Joseph Mulligan, … Anna Tompsett

Avoid common mistakes on your manuscript.

Introduction

The United Nations (UN) sustainable development goals (SDG) are a universal political agenda that address for a collective action to achieve a better and more sustainable future for all, solving the social, economic, and environmental issues that hinder global progress towards sustainability intended to be achieved by the year 2030 (United Nations 2015 ).

The UN General Assembly approved the Resolution A/RES/70/1 on “Transforming our world: the 2030 Agenda for Sustainable Development.” The agenda outlined 17 SDG and specific targets and indicators for each of the 17 SDG were defined by UN, totalizing 169 targets and 213 indicators that form a global action plan (United Nations 2017 ). Furthermore, the Agenda established five areas of critical importance known as the five pillars (5Ps): people, planet, prosperity, peace, and partnership (Tremblay et al. 2020 ) (Table 1 ).

The SDG are a recognized blueprint essential to achieve shared and sustainable prosperity with global action among governmental and non-governmental organizations, businesses, industry, civil society organizations, research, and technology development (Khaled et al. 2021 ).

However, there are crucial challenges to overcome, emphasizing the importance of the interrelationship between sectors, actors, and countries that have lesser and greater economic development (Stafford-Smith et al. 2017 ). Additionally, it can be pointed out the strong interdependencies between the failure or delay to implement one goal and how it will have repercussion in the others goals (Randers et al. 2018 ; Díaz-López et al. 2021 ).

Consequently, due to these interrelationships complexity, it is valuable for researchers to assess the status of the SDG research, for instance by mapping the existing knowledge or creating new knowledge to contribute to achieve the goals defined by the United Nations and also allow the overcoming of previous partial approaches to sustainable development (Belmonte-Ureña et al. 2021 ; Bordignon 2021 ).

A simple query performed through Web of Science (WoS) using the keywords “Sustainable Development Goal*” performed on October 1, 2022 resulted in 37,937 records. This demonstrates the great interest in the SDG as an object of research in recent years. Given these numbers, it is relatively difficult to map and identify the status of SDG research because of their infinity. Other issues deal with subjectivity, transparency, and delay in the literature review process. A broad view of a research area is important for obtaining valuable and impartial prospects for future research developments. Thus, a comprehensive review is needed to facilitate the integration of the contributions to provide a critical perspective (Díaz-López et al. 2021 ).

Bibliometric analysis is a statistical technique applied to examine the scientific production in a field of research. It allows to study the evolution of knowledge on a given topic during a certain period of time based on data publication (Belmonte-Ureña et al. 2021 ; Zupic and Čater 2015 ). It combines two main procedures: (i) the performance analysis and (ii) science mapping. The performance analysis is established on indicators that provide data about the amount and impact of the research through the application of several techniques, as citation analysis, counting publications, word frequency analysis by a unit of analysis (Romanelli et al. 2021 ). Science mapping is a graphic representation of how different scientific elements (knowledge areas, documents or authors) are interrelated. It shows the impact, structural, and dynamic organization of a knowledge topic, a field of research, a group of researchers, or a document, based on relation indicators (Marzi et al. 2017 ; Pizzi et al. 2020 ). Science mapping allows finding insights into patterns of a knowledge area that would be difficult to identify using traditional research review methods (Hallinger and Chatpinyakoop 2019 ; Prieto-Jiménez et al. 2021 ). Furthermore, science mapping analysis can be used to show or uncover some invisible key elements in a specific interest area (Cobo et al. 2011 ).

Although many bibliometric studies have been found in the field of SDGs research, most of them focus on specific disciplines, such as SDG and the business sector (Pizzi et al. 2020 ), education (Prieto-Jiménez et al. 2021 ), poverty (Yu and Huang 2021 ), and few bibliometric studies cover the SDG general aspects and its evolution. Nevertheless, some interesting bibliometric studies of SDG trends are worth mentioning, such as the study of Díaz-López et al. ( 2021 ), Meschede ( 2020 ), Yeh et al. ( 2022 ), and Sianes et al. ( 2022 ). All of them included a large number of documents (thousands) and do none of them focused on literature review articles.

Thus, the aim of this study was to conduct a bibliometric analysis of literature reviews on SDG from 2015 to 2022 with the following specific objectives:

Get a perspective of the status and evolution on the scientific research of SDG

Provide a visual representation of interrelations of the SDG review articles and its scientific elements

Reveal insights from the identified patterns of thematic currents

Define research gaps and hidden key elements on the SDG and propose future research

Materials and methods

The analysis of scientific literature was performed using a bibliometric analysis and was conducted in three phases (Fig.  1 ). It is worth mentioning that the present study restricted the analysis to review articles, that capture a general view of SDG research, and allow to understand and identify the domain of knowledge, the development of theories and concepts, and the academic debates in SDG research without carrying out any new studies or exhaustive review of the literature, as the review articles examine and summarize the state-of-art on certain topics from the available literature (Meschede 2020 ).

figure 1

Phases of methodology

Search and data collection

Analyzed metadata used in this investigation were obtained from the Clarivate Analytics WoS core collection database of the Institute for Scientific Information (ISI, Philadelphia, PA). Documents were retrieved by searching (‘‘SUSTAINABLE DEVELOPMENT GOAL*’’ or ‘‘SDG*’’) in the field “TITLE”, on SCI-expanded collection and as filter “REVIEW ARTICLES.” The search was conducted on October 1, 2022 and was narrowed to documents with publications years after 2015, because of the adoption year of the SDG, resulting in 312 documents.

All available metadata (abstract, keywords, funding, author, authors’ affiliation information, year of publication, thematic area, journal) were downloaded as a CSV-file. The data were checked for debugging using Microsoft Excel software, and a thesaurus file was created. In the thesaurus file, the keywords were normalized, eliminating duplicities, unifying synonyms, and developing acronyms. For this purpose, the all keywords were included. The VOSviewer 1.6.18 software was selected for this phase, due to its remarkable visualization feature for bibliometric data and also because it is a freely available tool (Meschede 2020 ; Prieto-Jiménez et al. 2021 ; van Eck and Waltman 2010 ).

Performance analysis

In this step, the basic characteristics of retrieved documents was performed using a descriptive bibliometric analysis exploring: (1) publication output; (2) research area; (3) most productive sources; (4) most-cited documents; (5) most productive countries. For the analysis of the most relevant sources, the impact factors were obtained from the Journal Citation Reports (JCR) published in 2021 that assesses the journals performance via the SCIMago Journal Rank (SJR) indicator based on an average number of citations. Furthermore, the total citations (TC), the average number of citations per paper (AC), the normalized citations (NC), the average normalized citations (ANC), and Hirsch index (h-index) were also used to assess the citation impact and productivity of a document, authors, and/or sources.

Cluster analysis and visualization

The third step included the cluster analysis and visualization by mapping technique. The co-authorship, bibliographic coupling and keyword co-occurrence were selected to be used as indicators (Ferreira 2018 ; Santana et al. 2020 ). The maps were interpreted according to the generated weights and score attributes assigned to each cluster (Garrigos-Simon et al. 2018 ; Prieto-Jiménez et al. 2021 ). An interpretive analysis was used to explore the conceptual structure of the SDG field and identify the thematic currents. Unique and significant keywords were identified in the co-occurrence mapping analysis. Thus, transversal terms (e.g., “sustainability”, “sustainable development goals” or “agenda 2030”) were excluded, as they can be associated with several SDG or the entire sustainable development agenda. In addition, keywords relating to specific methodologies (e.g., “review”, “bibliometric analysis”, “cluster analysis” or “systematic review”) were also excluded according to previous reported methodologies (Yakovleva and Vazquez-Brust 2012 ; Pizzi et al. 2020 ; Belmonte-Ureña et al. 2021 ).

Results and discussion

Publication output.

Figure  2 shows the cumulative distribution of scientific production of documents output of SDG reviews from its first issue in 2015 to 2022, after 7 years of its conception. In general, the publication output seems to increase over the years. In 2015, only one review was published, related to SDG 3, and the number of published review papers increased to 9 in the year of 2016. From 2017 to 2018, the number of published review papers per year increased dramatically from 10 articles to 40. Three years later, the number of published papers per year doubled and from then onwards, continues to grow, achieving a total of 312 publications in October 2022.

figure 2

Cumulative scientific production of review documents on sustainable development goals

Research areas

The cumulative publications of the main fields covered by literature reviews of SDG classified by WoS thematic categories published over the years are also shown in Fig.  2 . The main fields covered were (i) environmental sciences, (ii) green sustainable science technology, (iii) environmental studies, (iv) public environmental occupational health, (v) water resources, and (vi) energy fuels. The first category represents 44% of the total documents analyzed and it is of particular relevance. This result was expected, as environmental sciences is an interdisciplinary academic field that integrates physics, biology, and geography that studies the environment and the solution of environmental issues and includes a wide range of subjects (Zhu et al. 2021 ). Also, the publications could be included in multiple categories. Most of the documents (56%) were categorized in general aspects of sustainability, such as environmental sciences, environmental studies, and green and sustainable science technology. The others thematic categories that stood out were attributed to the relevance of SDG 3 and to technological themes that include SDG 6 and SDG 7, respectively.

In order to assess the distribution of SDG in literature review publications over the years, the articles were analyzed individually and classified as shown in Fig.  3 . An initial concern with SDG 3 was observed in relation to the first few years, since the first single publication in 2015, was referred to public environmental occupational health, and consequently with 100% of publications. A decrease trend in the proportion of SDG 3 publications over years was observed. It is clear that publications related to SDG 3 increased in number; however, the proportion of articles concerned to SDG 3 decreased, while the number and proportion of other SDG increased.

figure 3

Sustainable development goals (SDG) distribution of the review publications over the years

In the second year, an interest in relation to SDG 2, 12, and 17 was observed. Only in 2017 review articles of SDG related to hunger, energy, and peace were found. From 2018 onwards, more applied and diversified research were observed. Thus, areas related to nature and technology were covered, as water and sanitation, industry, innovation and infrastructure, food and agriculture, business and management, development studies and urban studies, climate change, life on land and water, are also slowly growing, indicating a trend in research in the environmental area. In addition, the results showed that the social area was the least studied, since only 2 and 4 review articles were found with regard to SDG 10 and SDG 5, respectively. The SDG are integrated, indivisible, and mixed, in a balanced way, in the three dimensions of sustainable development: economic, social, and environmental. Therefore, it will not be possible to achieve the goals if there is no balance between them (United Nations 2015 ).

Furthermore, it was observed that even though the year 2022 has not yet ended, this was the only year that presented review articles that includes all the SDG. This result also shows the trend of more diversified research on SDG.

The most relevant journals regarding the largest number of citations are presented in Table 2 . Among the top ten journals, most of them are related to general aspects of sustainability, such as, the journals: Current Opinion in Environmental Sustainability, Science of the Total Environment, Journal of Cleaner Production, Sustainability and Sustainability Science. Most journals are in regard to the general aspects of sustainability, according to the previous results. The journal Lancet and Globalization and Health are related to SDG 3, whereas Nature energy and Renewable Sustainable Energy Reviews are attributed to SDG 7. These results are in accordance to the results previously discussed.

Regarding the number of documents by journal, Sustainability has the largest number of reviews on SDG, concentrating 40 (13.8%) of all the articles analyzed (312), followed by the Journal of Cleaner Production with 4.2% participation of the review articles (13), and Current Opinion in Environmental Sustainability journal with 3.8% of the total dataset, that is also the most-cited journal with 790 total citations (TC). Concerning the impact factor, two journals stand out, the 79-point journal Lancet and the 61-point journal Nature Energy. In respect to the h-index, the Lancet (807) and Nature Communications (410) are the most remarkable. Similar results were obtained by Sianes et al. ( 2022 ) who used bibliometric methodologies to evaluate the impacts of SDG on the academic agenda.

The average number of citations per document (AC) show that Nature Energy, Nature Communications, and the Lancet stand out with 372, 147, and 110 citations, respectively. However, when observing the normalized citation (NC), that is the number of citations of a document equals the number of citations of the document divided by the average number of citations of all documents published in the same year included in the data that is provided to VOSviewer (van Eck and Waltman 2010 ), showed that Science of the Total Environment (NC = 27), Sustainability (NC = 27), and the Journal of cleaner production (NC = 21) are the most prominent sources. This normalization corrects the fact that older documents have more time to receive citations than recent documents (van Eck and Waltman 2010 ). Nonetheless, the average normalized citation (ANC), which indicates the NC divided by the number of documents, demonstrated that Nature Energies (ANC = 6.03), Nature Communications (ANC = 5.43), and the Science of total environment (ANC = 3.35) are the most noticeable journals.

Thus, it is noted that even though the total number of documents, TC, h-index, and IF indicate that SDG 3 is among the top 3 most relevant sources, the normalized citation data indicate that journals research on energy, multidisciplinary and general environmental sciences, and sustainability are among the most noticeable sources. It is important to highlight that this result is contrary to that obtained in bibliometric studies reported previously (Meschede 2020 ; Yeh et al. 2022 ). This indicates that the number of citations alone is not adequate to assess the relevance of a document or journal (Simko 2015 ).

One can observe in Table 3 , which show the most relevant review documents based on the TC of the documents, that the majority are focused on SDGs 3 and 7. In this way, review publications go even further in determining strategies and methodologies that seek to achieve the SDG through serving the health and energy sectors. However, when analyzing the NC, the document that really stands out is related to SDG 9, followed by the SDGs 7 and 3, which corresponds to innovation, energy and mental health, respectively.

Furthermore, the density analysis of bibliographic coupling of the most-cited documents is represented in Fig.  4 . For this purpose, a minimum of 148 citations from each document were used as a limit, in order to obtain the top 10 most-cited documents. Thus, 11 documents met the stipulated threshold, as expected, and 4 clusters were formed with 21 links, and total link strength of 56, where the weights were the number of citations. In general, the documents did not present high coupling strengths, as the number of citations is represented by the intensity of the red color and the bibliographic coupling by the proximity of the documents. This behavior was expected, as those papers have distinct themes and their specific attention contribute to SDG and novelty on different areas.

figure 4

Density diagram of bibliographic coupling of documents (minimum number of citations of a document of the 312 documents: 148, meet the threshold: 11, clusters: 4, links: 21, total link strength: 56, weights: citations, VosViewer 1.6.18)

Nerini et al. ( 2018 ) mapped the synergies and trade-offs in energy and SDG, and Vinuesa et al. ( 2020 ) described about the importance of artificial intelligence. Both documents discussed about technology and SDG, and, thus, they present some coupling strength. However, McCollum et al. ( 2018 ) also discussed about energy (SDG 7) and did not present bibliographic coupling with those articles. These results of bibliographic coupling demonstrated the lack of integration between the research, even when within the same field.

Caiado et al. ( 2018 ) and Biermann et al. ( 2017 ) discuss similar themes of constraints and governance. Therefore, it was expected them to have high coupling strengths. Vanham et al. ( 2018 ) was the only article about the SDG 6 included in the most-cited reviews, which contains water scarcity metrics for monitoring progress, and thus, it was not found bibliographic coupling with others top-cited documents.

Buse and Hawkes ( 2015 ), Bekker et al. ( 2018 ), and Nugent et al. ( 2018 ) focused their articles on SDG 3 and reviewed studies about heath, HIV, and disease prevention, respectively. It is worth mentioning that the article of Lund et al. ( 2018 ), which brings a review study on social determinants of mental disorders, did not presented bibliographic coupling with the other documents of SDG 3. This is an indicative that the psychiatry field is traditionally separated from other medicine branches of medicine and health fields (Fiorillo and Maj, 2018 ), even though mental health should be considered as good health and well-being (SDG 3). These results demonstrate that the areas are not very interconnected even when dealing with the same subject.

In order to represent the importance of SDG review production by the most productive countries based on corresponding authors affiliations country, a total of 88 countries, from all continents, of the place of the scientific production analyzed by country or region were represented (Fig.  5 ). The United Kingdom was the most prolific region, accounting for 21.7% of the review articles (68), followed by the USA and Australia, which represented 19.6% (61) and 16.0% (50), respectively. Finally, India accounted for 10.9% (34) of the scientific production, followed by Germany and China, both accounting with 8% (25) each.

figure 5

The geographical distribution of documents

To better understand international collaborations, the mapping of co-authorship of countries according to the authors’ affiliations was presented in Fig.  6 . In addition, the citation characteristics of the clusters were provided in Table 4 . All countries that presented at least 5 publications were included. This condition was true for 34 countries, and 5 clusters were obtained with 310 links, and total link strength of 758. The size of the circle reflected the number of publications (weights) in the dataset that were associated to the country.

figure 6

Clusters based on co-authorship of countries according to the authors affiliations (minimum number of documents of a country: 5; items that meet the threshold from a total of 88 countries: 34; clusters: 5; links: 310; total link strength: 758; weights: documents; VOSviewer 1.6.18.) https://tinyurl.com/2m5sdem6

The first cluster was composed practically exclusively of European countries that included Italy, Austria, Ireland, Sweden, Spain, Germany, Belgium, the Netherlands, France, and Singapore. This cluster obtained the higher TC, AC, and ANC, despite it not presented the largest number of documents. An interesting result was that the highest value of ANC was for the only non-European country, Singapore (ANC = 2.48), even with only 5 documents published.

The second cluster consisted of multicontinental cooperation, Oceania, Africa, America, and Asia, composed of the countries, New Zealand, Kenya, Brazil, Pakistan, China, Canada, Malaysia, Bangladesh, and Thailand. The European continent was the only excluded in this cluster. The cluster 2 presented higher ANC when compared to cluster 3, which presented the same number of countries, a greater number of documents and citations, but a smaller percentage of developing countries. This is a strong indicative that developing countries are relevant when it comes to SDG, since there is an exclusive SDG that seeks equality (SDG 10) and this topic remains at the center of global debates (Díaz-López et al. 2021 ; Bose and Khan 2022 ).

The third red central cluster involves England and the USA, the most occurring countries. They are linked to almost all countries. However, they have stronger collaboration with United Arab Emirates, Denmark, Japan, India, Scotland, Norway, and Nepal. Despite the USA and England being among the countries with the highest number of documents, this cluster has the AC and ANC among the least cited, indicating that the large number of documents do not reflect the real impact of the review publications related to the SDGs (Bennich et al. 2020 ).

The fourth cluster comprises South Africa, Mexico, Switzerland, and Nigeria. And again, it shows that documents become more impactful and relevant when there is intercontinental cooperation, as this cluster comprises few countries and documents, but have high AC and ANC. In low- and middle-income countries, limited or non-existent resources are the great challenge for the development of research in all areas of knowledge. On the other hand, the reverse innovation approach (Harris et al. 2020 ) has increasingly encouraged rich countries to research in poor countries. In this context, low-resource environments become an opportunity for study and learning to share with rich countries, generating promising options for new products, processes, or policies, which, by different contexts, can provide solutions to gaps or unresolved challenges (Bhattacharyya et al. 2017 ).

Australia was the third most productive country (Fig.  5 ) and thus, was also linked to almost all analyzed countries. Whereas, Australia is linked especially to South Korea, the fifth cluster entails only two countries with cooperation of 5 documents. Furthermore, South Korea, due to having only link to Australia, presented among the lower ANC. This result is a strong indication that articles produced without internationalization are less relevant (Vrontis and Christofi 2021 ).

The analysis of keywords are essential in reflecting and defining the research contents and can be used to identify the framework of a main research in many areas (Wuni et al. 2019 ). The most relevant keywords co-occurrence were used to investigate the conceptual structure of the SDG literature reviews to identify the prevailing thematic (Pizzi et al. 2020 ). Figure  7 and Table 5 present the results of the co-occurrence analysis of the diversity of 1933 of all keywords from 312 documents analyzed with 6 as a minimum number of occurrences of a keyword, and no minimum number of citations and documents. The map resulted in 93 keywords that met the threshold with 6 clusters, 1250 links, and total link strength of 2036. The size of the circle reflected the number of occurrences (weights) of each keyword in the dataset.

figure 7

Clusters based on co-occurrence of all keywords (minimum number of occurrences of a keyword: 6; items that meet the threshold: 93; clusters: 6; links: 1250; total link strength: 2036; weights: occurrences; maximum lines: 600, VosViewer 1.6.18) https://tinyurl.com/2gfbs3hk

Analyzing the corresponding keywords of the review articles in the co-occurrence map a big cluster was obtained, which, despite being divided into six main clusters, they are very interconnected, indicating that all the keywords are related.

The first cluster is mainly associated with the thematic core of social issues, such as poverty, public health, education, gender equality, and reduced inequalities which reflects to the SDG 1, 3, 4, 5, and 10, respectively. The most frequent keywords in this cluster were policy, Africa, education, poverty, diseases and covid-19 with 28, 28, 16, 15, 12, and 12 occurrences, respectively. However, regarding their ANC, the most influential keywords was middle-income countries . Furthermore, despite this cluster having the largest number of keywords, its ANC is among the smallest. Thus, this is a strong indicative that social issues are not taken as seriously as those dealt with in other clusters.

The second cluster is divided in two main hubs, one for economic growth (SDG 8), production (SDG 12) and innovation (SDG 9) with the keywords Industry 4.0, big data, supply chain , technology, corporate social-reasonability, innovation, management, business , governance, circular economy; and one for climate action (SDG 13) and energy (SDG 7), with the keywords energy, impacts, climate . It is clear that energy and climate are well connected, as the growth of some industrial sectors directly affects the consumption and production of energy and consequently the impacts on the climate. Investigating the normalized words, a hidden key element was observed, the keyword industry 4.0 presented the highest ANC (3.99), which is 3.5 times higher than the average ANC of the analyzed keywords, followed by the keywords big data , supply chain , nexus , strategy , circular economy and technology .

Previous bibliometric studies have reported that research related to the SDGs is focused mainly on health (SDG 3), climate change (SDG 13), food (SDG 2), energy (SDG7), and sanitation (SDG 6) (Meschede 2020 ; Belmonte-Ureña et al. 2021 ; Yeh et al. 2022 ; Londoño-Pineda and Cano 2022 ; Sianes et al. 2022 ). Controversially, the present research found that academic research on SDGs 8, 9, 11, which was considered poorly researched and lacking in consolidated literature (Pizzi et al. 2020 ), obtained greater evidence, despite the number of occurrences and citations that can lead to an erroneous conclusion. This result is very important, as it is possible to observe the rapid growth and the importance of technology to achieve the goals, aiming at profitability and economic growth, in addition to pursuing environmental benefits (Walsh et al. 2020 ).

The third cluster is composed of a miscellaneous of SDG. However, regarding the occurrences of keywords, the keyword health (SDG 3) is central and is the most used keyword in this cluster with 37 occurrences, followed by water (SDG 6), cities (SDG 11), indicators , and sanitation (SDG 6). Furthermore, in regard to the ANC of the keywords, life-cycle assessment (SDG 12), waste management (SDG 11), tourism (SDG 8), industry (SDG 9) presented the higher scores. This result indicates that the most influential keywords in this cluster are related to economic growth, industry, innovation, and production, which are also linked to health conditions and the reduction of impacts on the planet, explaining the occurrence of the keywords, such as emission (SDG 13) and environmental impacts that are also part of this cluster (Rosa and Hassmiller 2020 ).

The keywords food security, agriculture , and food are the most used keywords in the fourth cluster and are associated with SDG 2. Bioenergy (SDG 7) and waste-water (SDG 6) are also present in the fourth cluster and can be also linked to the agricultural and/or food production. Some keywords could be attributed to SDG 15, such as land and soil . However, they possibly refer to land cultivation and therefore, are aiming at higher agricultural productivity, and thus, they were also classified as SDG 2. Furthermore, trade-offs and stakeholders were among the most evident keywords according to the ANC, and are related to the difficulties of implementing the SDG, being the only cluster where the 5Ps category partnership was found (SDG 17). This result indicate that applied research aiming at the implementation of the SDG is being developed.

The most prominent keyword in the fifth cluster is climate-change , with 53 occurrences, followed by ecosystem, adaptation, biodiversity , and gender . This cluster is well-defined and its issues can be assigned mainly to SDGs 13 and 15, and consequently to the 5Ps category, planet. Furthermore, regarding the ANC, it was noted that this cluster presented the lowest ANC, indicating the inferior relevance of the topic among the review articles studied.

No reviews articles related to marine water (SDG 14) were found (with a minimum number of 6 occurrences). Some authors (Bordignon 2021 ) also include freshwater in SDG 14, as being “life below water.” However, in the official document, the 2030 agenda only includes marine life (United Nations 2015 ). Many authors disagree with this distribution, which also causes some disagreements in the classification of SDG 6 on 5Ps, as people or planet. Tremblay et al. ( 2020 ) addressed this issue in his paper and proposed a relative distribution of the 5 Ps among the 17 SDGs according to the targets, whereas for SDG 6, about 40% corresponded to planet and 31% to people. Therefore, the lack of review articles on SDG 14 indicates a research gap in this field.

The sixth and last cluster includes the keywords, Electricity, greenhouse-gas, emissions, renewable energy, barriers, developing-countries that can be attributed to the SDGs 7, 10, and 13. International collaborative studies between developing countries and high-income countries were reported previously (Fig.  6 ). These collaborations are important for understanding the dynamics that affect developing countries due to the mixed and complicated impacts on achievement of SDG (Sianes et al. 2022 ). In this cluster, the most frequent keyword was developing-countries ; however, the ones with the highest ANC were electricity and greenhouse-gas emission , while developing-countries was the one with the lowest ANC, indicating assuredly that the number of citations is not an adequate method for measuring subject relevance (Simko 2015 ).

In addition, the SDG 16, peace, justice, and strong institutions , showed to be a major gap in the research on SDG literature reviews. This topic is directly related to society, policies and governments, and plea for global peace. The targets also aim to reduce violence, corruption, bribe, exploitation, trafficking, torture, abuse, illicit arms, organized crime, and also develop international cooperation, participatory, and inclusive decision-making, inclusive (United Nations 2015 ).

Armed conflicts pose serious threats within the reach of the SDGs. As an example, the Russian-Ukrainian armed conflict is a dramatic world event that, in addition to the loss of life, impacts the environment, economy, and society. Both in countries directly involved, as well as in other countries, especially in developing countries, which are more vulnerable to the economic crisis. The negative regional and global impact could weaken the ability of many nations to achieve the SDGs by 2030, and could even make them unattainable (Pereira et al. 2022 ).

Conclusions

This bibliometric analysis provided the perspective of the status and evolution of research trend in the domain of SDG, with special emphasis on literature reviews regarding the fulfillment and consolidation of the agenda 2030. The revealed conclusions were:

An increasing trend in publications of SDG literature reviews was observed with a growing diversification in research areas.

Most of the review documents were categorized in general aspects of sustainability. In addition, most of relevant sources were associated with energy, multidisciplinary, and also general aspects of sustainability.

The most-cited documents and the most relevant sources indicated that the SDG with the greatest relevance was related to artificial intelligence (SDG 9) according to normalized citation analysis.

The results of bibliographic coupling demonstrated the lack of integration between the research, even when within the same research area.

England, the USA, and Australia were the most productive countries in SDG review articles. However, countries with intercontinental collaboration and with collaboration with developing countries showed greater relevance.

The terms with the highest values of keywords occurrences were health (SDG 3), climate change (SDG 13), food (SDG 2), energy (SDG 7), and water (SDG 6).

The keywords co-occurrence analysis spotted technology (SDG 9) and economic growth (SDG 8) as hidden key elements and with the greatest relevance among the analyzed keywords.

An imbalance between the different SDG was observed among academic research, and the main gaps found were for literature reviews on reduced inequalities (SDG 10), gender equality (SDG 5); oceans, seas, and marine environments (SDG 14); and related to peace, justice, and strong institutions (SDG 16).

Finally, the analysis revealed that the proposed methodology using bibliometric analysis of literature reviews presented different results when compared to previous reports in which a large number of documents was analyzed. The SDG is highly complex due to the many targets involved, trade-offs, and its multidisciplinary aspects. Therefore, future research should be directed to fulfill this gap through the development of more effective and reliable methods to verify the evolution of SDG in each area of research. Furthermore, the conceptualization and implementation of a theoretical framework that can be used in a generic way and also that describe standalone research areas to analyze the SDG considering their complexity will be also needed. In addition, it was observed that much research is still very disciplinary. Therefore, another future opportunity observed was the inclusion of more direct inter-linkages between certain SDG and practical applications. Thus, all research should be associated with the SDG, so that the efforts would be focused on achieving the agenda 2030, and a paradigm shift, practical, interdisciplinary actions with a system thinking perspective are necessary.

Data availability

Raw data are available upon request.

Allen C, Metternicht G, Wiedmann T (2018) Initial progress in implementing the Sustainable Development Goals (SDGs): a review of evidence from countries. Sustain Sci 13:1453–1467. https://doi.org/10.1007/s11625-018-0572-3

Article   Google Scholar  

Bekker L, Alleyne G, Baral S et al (2018) Advancing global health and strengthening the HIV response in the era of the Sustainable Development Goals: the International AIDS Society-Lancet Commission. Lancet 392:312–358. https://doi.org/10.1016/S0140-6736(18)31070-5

Belmonte-Ureña LJ, Plaza-Úbeda JA, Vazquez-Brust D, Yakovleva N (2021) Circular economy, degrowth and green growth as pathways for research on sustainable development goals: a global analysis and future agenda. Ecol Econ 185:107050. https://doi.org/10.1016/j.ecolecon.2021.107050

Bennich T, Weitz N, Carlsen H (2020) Deciphering the scientific literature on SDG interactions: a review and reading guide. Sci Total Environ 728:138405. https://doi.org/10.1016/j.scitotenv.2020.138405

Bhattacharyya O, Wu D, Mossman K et al (2017) Criteria to assess potential reverse innovations: opportunities for shared learning between high- and low-income countries. Glob Health 13:4. https://doi.org/10.1186/s12992-016-0225-1

Biermann F, Kanie N, Kim R (2017) Global governance by goal-setting: the novel approach of the UN Sustainable Development Goals. Curr Opin Environ Sustain 26–27:26–31. https://doi.org/10.1016/j.cosust.2017.01.010

Bordignon F (2021) Dataset of search queries to map scientific publications to the UN sustainable development goals. Data Brief 34:106731. https://doi.org/10.1016/j.dib.2021.106731

Article   CAS   Google Scholar  

Bose S, Khan HZ (2022) Sustainable development goals (SDGs) reporting and the role of country-level institutional factors: an international evidence. J Clean Prod 335:130290. https://doi.org/10.1016/j.jclepro.2021.130290

Buse K, Hawkes S (2015) Health in the sustainable development goals: ready for a paradigm shift? Glob Health 11:13. https://doi.org/10.1186/s12992-015-0098-8

Caiado R, Leal W, Quelhas O et al (2018) A literature-based review on potentials and constraints in the implementation of the sustainable development goals. J Clean Prod 198:1276–1288. https://doi.org/10.1016/j.jclepro.2018.07.102

Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F (2011) Science mapping software tools: review, analysis, and cooperative study among tools. J Am Soc Inform Sci Technol 62(1382):1402. https://doi.org/10.1002/asi.21525

Díaz-López C, Martín-Blanco C, De la Torre Bayo JJ et al (2021) Analyzing the scientific evolution of the Sustainable Development Goals. Appl Sci 11:8286. https://doi.org/10.3390/app11188286

Ferreira FAF (2018) Mapping the field of arts-based management: bibliographic coupling and co-citation analyses. J Bus Res 85:348–357. https://doi.org/10.1016/j.jbusres.2017.03.026

Fiorillo A, Maj M (2018) The role of psychiatry in modern medicine. Int Rev Psychiatry 30:169–175. https://doi.org/10.1080/09540261.2018.1436675

Garrigos-Simon FJ, Narangajavana-Kaosiri Y, Lengua-Lengua I (2018) Tourism and Sustainability: a bibliometric and visualization analysis. Sustainability 10:1976. https://doi.org/10.3390/su10061976

Hallinger P, Chatpinyakoop C (2019) A bibliometric review of research on higher education for sustainable development, 1998–2018. Sustainability 11:2401. https://doi.org/10.3390/su11082401

Harris M, Dadwal V, Syed SB (2020) Review of the reverse innovation series in globalization and health – where are we and what else is needed? Glob Health 16:26. https://doi.org/10.1186/s12992-020-00555-6

Khaled R, Ali H, Mohamed EKA (2021) The Sustainable Development Goals and corporate sustainability performance: mapping, extent and determinants. J Clean Prod 311:127599. https://doi.org/10.1016/j.jclepro.2021.127599

Londoño-Pineda AA, Cano JA (2022) Assessments under the United Nations Sustainable Development Goals: a bibliometric analysis. Environ Clim Technol 26:166–181. https://doi.org/10.2478/rtuect-2022-0014

Lund C, Brooke-Sumner C, Baingana F et al (2018) Social determinants of mental disorders and the Sustainable Development Goals: a systematic review of reviews. Lancet Psychiatry 5:357–369. https://doi.org/10.1016/S2215-0366(18)30060-9

Marzi G, Dabić M, Daim T, Garces E (2017) Product and process innovation in manufacturing firms: a 30-year bibliometric analysis. Scientometrics 113:673–704. https://doi.org/10.1007/s11192-017-2500-1

McCollum D, Echeverri L, Busch S, et al (2018) Connecting the sustainable development goals by their energy inter-linkages. Environ Res Lett 13:033006. https://doi.org/10.1088/1748-9326/aaafe3

Meschede C (2020) The Sustainable Development Goals in Scientific literature: a bibliometric overview at the meta-level. Sustainability 12:4461. https://doi.org/10.3390/su12114461

Nerini F, Tomei J, To L et al (2018) Mapping synergies and trade-offs between energy and the Sustainable Development Goals. Nat Energy 3:10–15. https://doi.org/10.1038/s41560-017-0036-5

Nugent R, Bertram M, Jan S et al (2018) Investing in non-communicable disease prevention and management to advance the Sustainable Development Goals. Lancet 391:2029–2035. https://doi.org/10.1016/S0140-6736(18)30667-6

Pereira P, Zhao W, Symochko L et al (2022) The Russian-Ukrainian armed conflict will push back the sustainable development goals. Geogr Sustain 3:277–287. https://doi.org/10.1016/j.geosus.2022.09.003

Pizzi S, Caputo A, Corvino A, Venturelli A (2020) Management research and the UN sustainable development goals (SDGs): a bibliometric investigation and systematic review. J Clean Prod 276:124033. https://doi.org/10.1016/j.jclepro.2020.124033

Prieto-Jiménez E, López-Catalán L, López-Catalán B, Domínguez-Fernández G (2021) Sustainable Development Goals and education: a bibliometric mapping analysis. Sustainability 13:2126. https://doi.org/10.3390/su13042126

Randers J, Rockstrom J, Stoknes PE, Golüke U, Collste D, Cornell S (2018) Transformation is feasible: how to achieve the sustainable development goals within planetary boundaries. A report to the Club of Rome from Stockholm Resilience Centre and BI Norwegian Business School, Stockholm University.  https://www.stockholmresilience.org/download/18.51d83659166367a9a16353/1539675518425/Report_Achieving%20the%20Sustainable%20Development%20Goals_WEB.pdf

Romanelli JP, Gonçalves MCP, de Abreu Pestana LF et al (2021) Four challenges when conducting bibliometric reviews and how to deal with them. Environ Sci Pollut Res 28:60448–60458. https://doi.org/10.1007/s11356-021-16420-x

Rosa WE, Hassmiller SB (2020) The Sustainable Development Goals and building a culture of health. AJN Am J Nurs 120:69–71. https://doi.org/10.1097/01.NAJ.0000668772.33792.1f

Santana M, Morales-Sánchez R, Pasamar S (2020) Mapping the link between corporate social responsibility (CSR) and human resource management (HRM): how is this relationship measured? Sustainability 12:1678. https://doi.org/10.3390/su12041678

Sianes A, Vega-Muñoz A, Tirado-Valencia P, Ariza-Montes A (2022) Impact of the sustainable development goals on the academic research agenda. A scientometric analysis. PLOS One 17:e0265409. https://doi.org/10.1371/journal.pone.0265409

Simko I (2015) Analysis of bibliometric indicators to determine citation bias. Palgrave Commun 1:1–9. https://doi.org/10.1057/palcomms.2015.11

Stafford-Smith M, Griggs D, Gaffney O et al (2017) Integration: the key to implementing the Sustainable Development Goals. Sustain Sci 12:911–919. https://doi.org/10.1007/s11625-016-0383-3

Tremblay D, Fortier F, Boucher J et al (2020) Sustainable development goal interactions: an analysis based on the five pillars of the 2030 agenda. Sustain Dev 28:1584–1596. https://doi.org/10.1002/sd.2107

United Nations (2015) A/RES/70/1 - Transforming our world: the 2030 agenda for sustainable development. Resolution adopted by the General Assembly on 25 September 2015, p 35. https://documents-dds-ny.un.org/doc/UNDOC/GEN/N15/291/89/PDF/N1529189.pdf?OpenElement

United Nations (2017) A/RES/71/313 - Work of the Statistical Commission pertaining to the 2030 Agenda for Sustainable Development. Resolution adopted by the General Assembly on 6 July 2017, p 25. https://documents-dds-ny.un.org/doc/UNDOC/GEN/N17/207/63/PDF/N1720763.pdf?OpenElement

van Eck NJ, Waltman L (2010) Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 84:523–538. https://doi.org/10.1007/s11192-009-0146-3

Vanham D, Hoekstra A, Wada Y et al (2018) Physical water scarcity metrics for monitoring progress towards SDG target 6.4: an evaluation of indicator 6.4.2 “Level of water stress.” Sci Total Environ 613:218–232. https://doi.org/10.1016/j.scitotenv.2017.09.056

Vinuesa R, Azizpour H, Leite I, et al (2020) The role of artificial intelligence in achieving the Sustainable Development Goals. Nat Commun 11:233. https://doi.org/10.1038/s41467-019-14108-y

Vrontis D, Christofi M (2021) R&D internationalization and innovation: a systematic review, integrative framework and future research directions. J Bus Res 128:812–823. https://doi.org/10.1016/j.jbusres.2019.03.031

Walsh PP, Murphy E, Horan D (2020) The role of science, technology and innovation in the UN 2030 agenda. Technol Forecast Soc Chang 154:119957. https://doi.org/10.1016/j.techfore.2020.119957

Wuni IY, Shen GQP, Osei-Kyei R (2019) Scientometric review of global research trends on green buildings in construction journals from 1992 to 2018. Energy Build 190:69–85. https://doi.org/10.1016/j.enbuild.2019.02.010

Yakovleva N, Vazquez-Brust D (2012) Stakeholder perspectives on CSR of mining MNCs in Argentina. J Bus Ethics 106:191–211. https://doi.org/10.1007/s10551-011-0989-4

Yeh S-C, Hsieh Y-L, Yu H-C, Tseng Y-H (2022) The trends and content of research related to the Sustainable Development Goals: a systemic review. Appl Sci 12:6820. https://doi.org/10.3390/app12136820

Yu Y, Huang J (2021) Poverty reduction of Sustainable Development Goals in the 21st century: a bibliometric analysis. Front Commun 6:754181. https://doi.org/10.3389/fcomm.2021.754181

Zhu D, Chen W, Qu X et al (2021) Future research needs for environmental science in China. Geogr Sustain 2:234–242. https://doi.org/10.1016/j.geosus.2021.09.003

Zupic I, Čater T (2015) Bibliometric methods in management and organization. Organ Res Methods 18:429–472. https://doi.org/10.1177/1094428114562629

Download references

This work was supported by Cesumar Institute of Science, Technology and Innovation, Cesumar University (ICETI, Brazil).

Author information

Authors and affiliations.

Post-Graduation Program in Clean Technologies, Cesumar Institute of Science, Technology and Innovation, Cesumar University, Maringá, Brazil

Natália Ueda Yamaguchi, Eduarda Gameleira Bernardino, Maria Eliana Camargo Ferreira & Bruna Pietroski de Lima

Post-Graduation Program in Health Promotion, Cesumar Institute of Science, Technology and Innovation, Cesumar University, Maringá, Brazil

Mauro Renato Pascotini & Mirian Ueda Yamaguchi

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the study conception and design. Investigation, methodology, data collection, and analysis were performed by Natália Ueda Yamaguchi, Eduarda Gameleira Bernardino, Maria Eliana Camargo Ferreira, Bruna Pietroski de Lima, Mauro Renato Pascotini, and Mirian Ueda Yamaguchi. The first draft of the manuscript was written by Natália Ueda Yamaguchi and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Natália Ueda Yamaguchi .

Ethics declarations

Ethical approval.

Not applicable.

Consent to participate

Consent to publish, competing interests.

The authors declare no competing interests.

Additional information

Responsible Editor: Philippe Garrigues

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Yamaguchi, N.U., Bernardino, E.G., Ferreira, M.E.C. et al. Sustainable development goals: a bibliometric analysis of literature reviews. Environ Sci Pollut Res 30 , 5502–5515 (2023). https://doi.org/10.1007/s11356-022-24379-6

Download citation

Received : 27 July 2022

Accepted : 19 November 2022

Published : 24 November 2022

Issue Date : January 2023

DOI : https://doi.org/10.1007/s11356-022-24379-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Agenda 2030
  • Citation analysis
  • Health promotion
  • Scientometrics
  • Sustainability
  • Find a journal
  • Publish with us
  • Track your research

To read this content please select one of the options below:

Please note you do not have access to teaching notes, digital labour: a systematic literature review and bibliometric analysis.

International Journal of Organizational Analysis

ISSN : 1934-8835

Article publication date: 28 June 2023

This study aims to conduct a systematic literature review and bibliometric analysis on the topic of digital labour. The study also identifies the future research directions for the topic.

Design/methodology/approach

In total, 118 research papers were identified and reviewed from 11 established research databases and A*, A and B category journals from the ABDC journal list. The papers covered a timespan between 2006 and 2023. Bibliometric analysis was conducted to identify key research hotspots.

The emergent themes and associated sub-themes related to digital labour were identified from the literature. The paper found three significant themes that include digital labour platform, gig economy and productivity. This study also acts as a platform to initiate further research in this field for academicians, scholars, industry practitioners and policymakers. The future research scope in the topic is also presented.

Originality/value

The present study is unique in its nature as it approaches the topic of digital labour from all relevant perspectives.

  • Digital labour
  • Systematic literature review
  • Bibliometric analysis

Goel, V. , Unny, B.R. , Shome, S. and Gupta, Y. (2023), "Digital labour: a systematic literature review and bibliometric analysis", International Journal of Organizational Analysis , Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/IJOA-12-2022-3558

Emerald Publishing Limited

Copyright © 2023, Emerald Publishing Limited

Related articles

We’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Advanced Search
  • Journal List
  • Springer Nature - PMC COVID-19 Collection

Logo of phenaturepg

Systematic literature review and bibliometric analysis on virtual reality and education

Mario a. rojas-sánchez.

1 Costa Rica Institute of Technology, Cartago, Costa Rica

Pedro R. Palos-Sánchez

2 Department of Financial Economics and Operations, University of Sevilla, Seville, Spain

3 NECE-UBI Research Unit in Business Sciences, University of Beira Interior (UBI), Covilhã, Portugal

José A. Folgado-Fernández

4 Departamento de Economía Financiera y Contabilidad, Universidad de Extremadura, Badajoz, España

Associated Data

Data sets used and/or analyzed during the current study are available from the corresponding author under reasonable request.

The objective of this study is to identify and analyze the scientific literature with a bibliometric analysis to find the main topics, authors, sources, most cited articles, and countries in the literature on virtual reality in education. Another aim is to understand the conceptual, intellectual, and social structure of the literature on the subject and identify the knowledge base of the use of VR in education and whether it is commonly used and integrated into teaching–learning processes. To do this, articles indexed in the Main Collections of the Web of Science, Scopus and Lens were analyzed for the period 2010 to 2021. The research results are presented in two parts: the first is a quantitative analysis that provides an overview of virtual reality (VR) technology used in the educational field, with tables, graphs, and maps, highlighting the main performance indicators for the production of articles and their citation. The results obtained found a total of 718 articles of which the following were analyzed 273 published articles. The second stage consisted of an inductive type of analysis that found six major groups in the cited articles, which are instruction and learning using VR, VR learning environments, use of VR in different fields of knowledge, learning processes using VR applications or games, learning processes employing simulation, and topics published during the Covid-19 pandemic. Another important aspect to mention is that VR is used in many different areas of education, but until the beginning of the pandemic the use of this so-called “disruptive process” came mainly from students, Institutions were reluctant and slow to accept and include VR in the teaching–learning processes.

Introduction

The knowledge society recognizes that Education Institutions are a fundamental part of the globalization process, where the use of information and communication technologies (ICT) improve students' attitudes towards learning (Lazar & Panisoara, 2018 ). The concept of education refers to the process of facilitating learning, acquiring knowledge, skills or positive values with the aim of preparing students for life, work and citizenship (Kamińska et al., 2019 ). Virtual platforms often simulate the classroom and can provide a safe environment for testing experiments that can be dangerous in real life (Tzanavari & Tsapatsoulis, 2010 ). This revolutionizes learning processes, although professional training and scientific research are required, to facilitate innovative processes and develop new knowledge to meet the challenges of modern world demands (Castillo, 2010 ). The use of digital technologies has increased at all academic levels with educators adopting them in order to improve the learning experience of their students (McGovern et al., 2019 ).

Learning about sciences cannot always be fully implemented in classrooms, so it can be useful to use other options (Buehl, 2017 ; Folgado-Fernández et al., 2020 ). For example, scientific experiment in which physical risks would exist for students or use of very expensive scientific-technological material. In these cases, VR could simulate this environment and e-learning conditions. VR is suited to this as it consists of using a 3D environment that has been generated by a computer where the user can navigate and interact, achieving real-time simulation with a part, or all, of a user's senses (Guttentag, 2010 ). Another definition considers virtual reality as an immersive and interactive three-dimensional computer-generated environment in which interaction can occur on multiple sensory channels such as touch and position (Brey, 2014 ).

VR has been widely investigated in many fields such as sports (Vignais et al., 2015 ), tourism (Tussyadiah et al., 2018 ), virtual stores (Bonetti et al., 2018 ; Mann et al., 2015 ), healthcare from surgery simulation to nursing applications (Beyer-Berjot et al., 2016 ; Fagan et al., 2012 ), the military for flight simulation and training (Mihelj et al., 2014 ) and of course, in education (Zhang et al., 2017 ). Applications in education can be found for geography (Lv et al., 2017 ), nature sciences (Palos-Sanchez et al., 2022 ), mathematics (Xu & Ke, 2016 ), robotics (Román-Ibáñez et al., 2018 ), construction safety (Pham et al., 2018 ), medical training and assessment (Lövquist et al., 2012 ), physical education (Gómez-García et al., 2018 ) and many others.

Compared to traditional education, using virtual simulation-based training provides safety, cost savings and efficiency because less time is required for training (Shen et al., 2019a , b ). Educators' acceptance of VR in the classroom has been successfully investigated by (Hussin et al., 2011 ). The behavioral intentions for using VR in learning was analyzed (Shen et al., 2019a , b ) and also its acceptance by surgeons using VR for training (Hen, 2019 ).

The aim of this study is to identify and analyze the scientific literature with a bibliometric review to find the main topics, authors, sources, most cited articles and countries, as well as to know the conceptual, intellectual and social structure and identify the knowledge base of VR in education, and whether it is commonly used and integrated into teaching–learning processes. To reach the objectives, the articles in the scientific production indexed in the Web of Science and Scopus Main Collection were consulted, analyzing the articles and the emerging trends in research in articles published between January 1, 2010, and July 31, 2021.

This study analyzes relevant data from previous research to answer the following research questions (RQ) (Table ​ (Table1 1 ):

Research questions

This article is organized as follows: the first part introduces the study, including the objectives and the research questions. A second section presents a review of the literature on bibliometric analysis. The methodology used is defined in the third section, indicating the search procedures used to identify the literature on VR in education. The fourth section presents the results, and these are discussed in the fifth section. Finally, in the sixth section, the conclusions are drawn, and future lines of research are suggested.

Review of the literature on bibliometric analysis

This study uses a bibliometric analysis, which is a term that was coined by Pritchard ( 1969 ) who stated that it can be applied in all studies that aim to quantify the process of written communication (Gokhale et al., 2020 ).

Bibliometric analysis is an approach that uses a set of quantitative methods to measure, track, and analyze scholarly literature (Roemer & Borchardt, 2015 ). It identifies the publications by authors, the most prominent journals, as well as the methodologies used and the conclusions obtained (Durán Sánchez et al., 2014 ).

Metadata gives an overview of any field of research (Milian et al., 2019 ). Bibliometric methods involve a large volume of bibliographic material and have been used for the analysis of different topics (Blanco-Mesa et al., 2017 ), Journals (Martínez-López et al., 2018 ), Countries (Mas-Tur et al., 2019 ) and others.

The scientific literature contains important bibliometric analyses such as that by Huang et al. ( 2016 ), who performed a retrospective bibliometric analysis of articles about rehabilitation medicine using VR technology. The conclusion was that VR technology was one of the most popular technological advances. The results found a rapid growth in the production of articles in recent years.

Methodology

In this study, the selected dataset is analyzed using a quantitative exploration with a bibliometric study that identifies and analyzes the literature on VR in education to provide a map of the knowledge structure (Álvarez-García et al., 2019 ). A performance analysis and scientific mapping is done in the first part of the study. The scientific or bibliometric mapping provides a representation of how disciplines, fields, specialties, individual papers, and authors are related to each other (Small, 1999 ). The recommendations of (Cobo et al., 2011 ) were used to produce the maps to know the research topics and the different structures in the dataset.

A second analysis was also performed, ordering the articles in descending order according to the number of citations. The recommendations of (Heradio et al., 2016 ) were used to complete this part of the study. Figure  1 shows the methodology used. This figure shows different steps. For example, data cleaning was performed after applied inclusion criteria (see Table ​ Table2—Search 2 —Search string) or indicator elaboration permits sorted in relevance different articles.

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig1_HTML.jpg

Methodology used. Methodology according to (Danvila-del-Valle et al., 2019 )

Documents obtained from searches in the databases

Articles that constitute a representative sample of international scientific activity published in scientific journals were analyzed (Durán-Sánchez et al., 2018 ; Velasco et al., 2011 ). Therefore, meetings papers, editorials, books, chapters, proceedings, news, and other types of documents found in the databases were excluded.

Identification of sources

Data was gathered from journal articles indexed in the Web of Science Core Collection. This database was selected because of three criteria:

  • It has quality indexes such as JCR.
  • It covers a long time period.
  • It allows a considerable number of stored references to be downloaded simultaneously.

The presence of these characteristics is sufficient to justify its use (Durán-Sánchez et al., 2019 ).

The Scopus multidisciplinary bibliographic database was also used to find information in articles from scientific journals (ASJC) classified into an organized hierarchy of fields and subfields (Hassan et al., 2019 ). This database was selected because of three criteria:

  • It has quality indexes such as SJR.
  • It provides approximately 20% more coverage than the Web of Science.
  • Simultaneous downloading of a considerable number of references is allowed (Falagas et al., 2008 ).

We used the Lens database and academic meta-search engine, compatible with the biblioshiny software used for data analysis. The search was limited to articles containing the keyword “virtual reality” in the title. This is included in quotation marks to obtain all documents containing that combination of words in the document title and also containing the possible combinations with the terms educat* (to obtain words that start with that word but may have different endings). The terms learn*, teach* class* and student* were also used to obtain articles with titles containing words related to learning, teaching, class or classroom, students. innovat* and Covid* were also included to obtain articles about innovation and the teaching process during the Covid pandemic.

Study selection criteria

The search was performed in English to obtain the largest number of documents in the dataset on VR in education. Table ​ Table2 2 shows the initial data obtained with the proposed search strings. The inclusion criteria applied was document type: only articles were selected, language: English and years of publication: 2010–2021(July). The exclusion criteria were to exclude the field of Medicine. This study has not included specialized areas of medicine to get an overview of as many applications as possible in Education. It is particularly valuable for researchers, as information is presented about present and future lines of research which investigate the usefulness of VR in periods of crisis and confinement such as the one, we have just faced evidenced by the sudden leap in the use of technologies, and therefore virtuality, in platforms, applications, games and videos.

Data analysis process

The documents included in this analysis contain bibliographic information obtained after a manual review of the 436 relevant documents found in WoS, 584 found in Scopus and the 251 in Lens. 553 duplicate documents were eliminated, and the names of the authors and journals were normalized, which resulted in 718 documents unified in an.xlsx file. This whole process is summarised in the three stages of search strategies shown in Fig.  2 .

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig2_HTML.jpg

Search strategy summary. Note: Search strategy for the articles

Biblioshiny software was used for the analysis of the data. It is a tool which analyzes all the data identified in the body of literature and identifies the main themes (Huber, 2002 ). The application provides a web interface for the Bibliometrix software version 3.0 (Aria & Cuccurullo, 2017 ) and provides the data in graphical format, if desired, to visualize the statistics. In this study, the graphs describe the information about VR in the educational field over the time period chosen for the study. Figure  2 summarizes the search strategy used.

SLR articles on VR and education

Tables ​ Tables3 3 and ​ and4 4 show some summarized systematic literature reviews (SLR) on virtual reality and education. There has recently been an increase in the number of documents, possibly due to current circumstances like the Covid pandemic and teleworking, which means that technological innovation in education systems has greater prominence than before.

Summary of related systematic literature reviews

Main information

Main statistical indicators

The use of ICTs in education is commonplace, with VR no exception and often included in the teaching–learning processes. The evolution in the productivity of articles over the period analyzed clearly shows a rapid growth from 2015 to 2021, as shown in Fig.  3 .

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig3_HTML.jpg

Annual production of articles. Note: Annual productivity of virtual reality in education. Dotted line is the exponential trend line

This growth is due to the development of specific content in VR, with more and more sectors involved such as: real estate, locomotion, security, and even education itself with the new e-Learning systems.

Performance analysis

According to Heradio et al. ( 2016 ) the main procedure for research performance evaluation is citation analysis, which means, the more citations of an article, the greater its influence in that field. The h-index is considered a suitable measure of the quantity and impact of the scientific output of the publications of a researcher.

Overview of the analyzed data set

The information from the analyzed data is summarized in descriptive statistics presented in Table ​ Table4. 4 . Considering the results obtained, we can say that RV is a topic of great academic interest as evidenced by the number of papers (718) and the more than ten average citations per article.

The average number of annual citations are presented in Fig.  4 , while Fig.  5 provides an overview of the trends in the knowledge structure of the use of VR in education.

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig4_HTML.jpg

Average annual citations per year. Note: Average total number of citation per year

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig5_HTML.jpg

Trend tropics. Journals with most productivity and impact according to the h-index

Figure  4 shows that the highest average number of citations per year were in 2010 with 6.9 citations per year and in 2014 with 7.3. Contrary to what the authors expected due to the important advances and changes in the market for VR, there were only 2.55 citations in 2016 continuing with little growth until 2018 and maintaining lower averages thereafter. 2020 and the current, available data for 2021 does not show an impact on citations due to the covid-19 situation.

Figure  5 shows that the main topics of the trends from 2010 to 2017 were about using simulation as a learning tool to obtain greater student attention. 2018 continued integrating technology into the teaching process with topics such as the virtual community of players of 2nd life with customizable avatars that allow players to enjoy a second life. This uses voice text messaging with people from different places and countries and integrates visualization tools. In 2019 education included the design of technological environments. 2020 showed an increase in the literature consulted about e-learning using tools such as virtual or augmented reality. This could possibly be because of the changes in education methods imposed by the Covid-19 pandemic and the change from presential to virtual teaching, which was made abruptly in some cases. It should be noted that 6 months into 2021 the trend is towards platforms that can be used to teach or attend lessons, i.e., information technologies and user acceptance of these take on greater importance and there is also increasing interest in artificial intelligence with deep learning that uses machine learning processes such as speech recognition or automated translation.

The journal with the most impact in this study is Computers & Education with an h-index of 16. This means that a number, h, of publications of the journal have been cited h times. An h-index of 16 implies that this number of publications have been cited at least 16 times. Table ​ Table4 4 shows the journals ordered by the number of documents published, as well as the impact measured with the h-index. We can say that the selected journals contain 207 articles in total, of which 40% correspond to 3 publications: International Journal of Emerging Technologies in Learning, Computers & Education and Virtual Reality (Table ​ (Table5 5 ).

Magazines with most productivity and impact

Relevant information on the journals included in the dataset

Authors with most impact according to h-index

The authors with the highest productivity are shown in Fig.  6 and can be seen to be Chen W., Chen Y. and Lee J. Figure  6 orders the authors according to impact where it can be seen that Chen Y. and Jong M. have the highest impact with an h-index of 7, that is, each author has 7 papers with at least 7 citations each, which means that the author has been included in at least 49 publications (Fig.  7 ).

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig6_HTML.jpg

Most relevant authors

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig7_HTML.jpg

Impact of the authors

The most active authors in the last four years have been Chen W., Lee J., Kim, J., Ly Y. and Makransky G., as shown in Fig.  8 .

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig8_HTML.jpg

Productivity of the main authors over the period of time studied

Chen W. primarily studies problem solving in the classroom using VR technology, to assist in cognitive processing and knowledge transfer to the students. On the other hand, Lee J., studies the adaptation of the three-dimensional visualization made possible by immersive virtual reality.

Other authors, such as Kim J., jointly approach the analysis of VR and augmented reality (AR) whith the current skin electronics are summarized as one of the most promising device solutions for future VR/AR devices.

Ly Yan approaches the study of VR based simulation in hospital settings, that facilitates the acquisition of skills without compromising patient safety. Finally, Makransky's research deals with various aspects of RV in education, such as an important role in education by increasing student engagement and motivation.

The main affiliations of institutions can be seen in Fig.  9 , which shows National Taiwan Normal University as being the most productive with 17 papers published in the analyzed dataset. In second position is Chinese University Hong Kong with 12 papers and in third position is Texas Aandm University with 11 papers.

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig9_HTML.jpg

Most relevant affiliations

Main documents and most frequently used words in the dataset

Table ​ Table6 6 shows the documents with the largest number of citations in this study. The authors of the document with most citations were (Merchant et al., 2014 ) with 452 citations and, in second place, (Huang et al., 2010 ) with 236 citations, both of which were published in the Computer & Education journal. In third place was (Jensen & Konradsen, 2018 ) with 196 citations in the Telematics and Informatics journal. When an article has many citations, it influences the researchers who develop the area under investigation (Rodríguez & Navarro, 2008 ).

Most globally cited documents in the dataset

The words which occur with the highest frequency in the dataset can be seen in Fig.  10 . The first four words are related to the terms contained in the search strings, but the frequency and hierarchy follows the occurrences of the words “e-learning”, “environments”, “augmented reality”, “technology”, “simulation” and “learning systems”. This highlights the technological component of the field of education. The coincidence of keywords represents the knowledge structure of the literature (Cheng et al., 2018 ).

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig10_HTML.jpg

Most relevant words

Scientific mapping analysis

A similarity measure known as the strength of association was used to construct the bibliometric maps (Cobo et al., 2011 ; Van Eck & Waltman, 2007 ). This allows a variety of scientific maps to be prepared which show the structural and dynamic aspects of the data obtained from the scientific research (Börner et al., 2003 ).

According to Cobo et al. ( 2011 ) the maps show the evolution of a field of research and the conceptual structure of the field can be found from the co-occurrence. Co-citation and bibliographic coupling allow us to analyze the intellectual structure of a field of scientific research and the social structure can be found by analyzing the authors, also known as co-authorship analysis, as well as the data found from the author's affiliations such as the organization or country.

Main topics in keywords plus according to factor analysis

Figure  11 shows a two-dimensional graph formed by the topic words in Keywords Plus of the cited papers. A multiple correspondence analysis can be used to summarize big data with multiple variables in a low-dimensional space, creating a two-dimensional map where the words near the central point of the group have received a lot of attention in recent years and those near the edge are topics which have been used less in research or have been incorporated into other topics (Xie et al., 2020 ).

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig11_HTML.jpg

Factor map cluster analysis of high-frequency key words . Note: Factor analysis, keyword map, number of terms: 50, number of clusters: 2, label size: 12, number of words: 500

The first cluster covers words related to VR display devices which are used by users of virtual reality educational products. When it is not clear which device to use in a curriculum, the relevant constituent components of immersive technologies which differentiate their roles must be considered. An example is for the two common modes of virtual reality displays, head-mounted display (HMD) and desktop computer (DT) which may affect spatial learning (Srivastava et al., 2019 ). On the one hand desktop-based VR has higher installation costs, while mobile device-based virtual reality cannot produce the same environment quality due to the limited processing power. A result of the lower environment quality has, in some cases, caused higher rates of nausea and blurred vision (Moro et al., 2017 ).

A person can interact in an environment created with VR in a seemingly real or physical way by using special electronic equipment, such as a helmet with a screen inside it or gloves equipped with sensors (Katsioloudis et al., 2017 ). Jensen and Konradsen ( 2018 ) identify situations where HMDs are useful for cognitive skill acquisition, such as remembering and understanding spatial and visual information, psychomotor skills like head movement, visual or observational exploration and affective skills for emotional control and response to stressful or difficult situations.

The second group covers VR applied to education, which includes topics such as technologies that allow the visualization of situations, which receive more attention from students and motivate them. Examples of this type of technology are virtual communities, educational games, interactive learning environments, educational technologies that improve the teaching process, online learning, user experience, immersive learning, immersive virtual reality and deep learning. There is a wide variety of possibilities, most of which are immersive, using helmets, games or applications that provide an interactive learning experience for students.

Learning environments using animation and multimedia highlight a change in VR learning which is more immersive, simulating the real world with 3D models that provide an interactive environment and reinforce the feeling of immersion. Using this technology, educators combine theory and instruction methods that allow intelligent use of these environments (Huang et al., 2010 ). There are many ways to create these environments with equipment like VR helmets for experiential learning in a virtual space (Kwon, 2019 ) and new and improved environments, such as the PILE System that integrates video capture technology into the classroom where interaction is made through physical movements (Yang et al., 2010 ). As technology advances, better graphics and virtually animated actors or avatars can be used. These improve the applications by being more motivating and enjoyable, even though the applications become more complex which may prevent a novice learner from learning effectively (Kartiko et al., 2010 ).

VR applications enable potential learning. Authors such as Johnson-Glenberg, ( 2018 ) explore applications of educational theory which design classes using immersive virtual reality with two unique attributes of VR, which are making the student feel present in any given situation and to be able to use gestures and perform manipulations in three dimensions. For decades the primary interfaces of educational technology have been the mouse and keyboard, but now highly immersive environments can enhance learning and affect the way content is retained and encoded.

Games are useful in educational technology with many examples available. Some of these are used to train students in safety through role-playing and social interaction (Palos-Sanchez et al., 2018 ), which allows students to understand the causes of accidents and inspect risks in an immersive environment provided by the game (Le et al., 2015 ). Interaction was found to play an important role in understanding mathematics and geometry with problem solving. A whiteboard and a virtual tool were used to solve problems individually or in pairs. Group learning was found to be more effective, although the results of the groups were different as the difficulty of the problems were varied (Hwang & Hu, 2013 ).

Articles were found concerning online learning which shares digital content and technological tools for e-learning and virtual reality learning. One of these articles compares techniques such as email, attachments, shared use of Web interfaces and a VR engine which provides a virtual interface. The results indicate that users completed their workflow 50% faster with the VR option (Lampert et al., 2018 ). Another application that marked a change in e-learning is an innovative tool for young adults with mild cognitive impairments. It is an immersive virtual reality game called "In Your Eyes" that focuses on skills related to spatial perspective involving all five senses which shows that an immersive world can be an excellent training method (Freina et al., 2016 ).

Co-occurrence network mapping

Bibliometric mapping of the keywords used by the author was done to gain a thorough understanding of the conceptual structure. Apart from the Keyword Virtual Reality itself, it highlights those related to education, e-Learning and Students. The co-occurrence analysis is shown in Fig.  12 .

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig12_HTML.jpg

Keywords Plus co-occurrence network

Keyword co-occurrence analysis is an effective tool for understanding knowledge structures and research trends. This makes it easier to understand primary and secondary publications (Altınay Ozdemir & Goktas, 2021 ). In this figure, one should start by distinguishing nodes by their size. This represents the number of documents, while the line between two nodes represents a link between the two groups. A link means a co-occurrence between the two keywords (Guo et al., 2019 ). If the line is short the link is strong and vice versa.

In this bibliometric analysis we mainly distinguish the following keywords: 'virtual reality', 'e-learning' and 'students' in a first cluster. Each cluster represents a keyword and shows the most linked and repeated keywords in the publications. All clusters have a different colour. In Fig.  12 a distinction is made between the red colour for this cluster and the blue colour showing the following main keywords: 'Education', 'Technology', 'augmented reality', 'performance', 'simulation' and 'environments'. Because this bibliometric study found few papers, the number of co-occurrence links between keywords was not excessive. As Fig.  12 shows, two groups had a stronger relationship: 'Education Technology for simulation environments with augmented reality' and 'Virtual reality for e-learning systems'.

Productivity mapping of items by country

The countries or regions with the highest document productivity in this study are the Republic of China with 273, United States of America with 242 documents, followed by South Korea with 57 and Spain with 50, as shown in Fig.  13 . The high productivity of the United States is consistent with the bibliometric mapping for an analysis of studies on foreign language teaching in early childhood education by (Yilmaz et al., 2019 ) as well as the work of other authors (Hernández-Torrano & Kuzhabekova, 2019 ). China is a world power in VR technology and Chinese universities have been concerned to increase research in this field to meet the challenges posed by VR technologies. The main reason is that Chinese people are very prone to adopt emerging technologies, we can say that it is an important virtual reality market in the world.

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig13_HTML.jpg

Productivity of items by country

In Fig.  14 we see how the main keywords, students, virtual reality, technology, e-learning, have a greater relationship with the countries of China and the USA, as well as the universities in the last column, which reflects a greater scientific production in topics related to technology applied to education.

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig14_HTML.jpg

Three fields plot

VR technology in the educational field is opening up space through e-learning, game-based learning to mobile learning, going from simulation, machine learning to Deep Learning, where immersive virtual reality is part of the topics present as shown in Fig.  15 .

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig15_HTML.jpg

Thematic evolution

In Fig.  15 we can see the thematic evolution through a Sankey energy diffluence diagram, which is a specific type of flow diagram. In this paper, based on the Sankey diagram, we visualise the thematic evolution over time in the field of VR and Education research. This figure helps us to understand the temporal evolution of the conditions in which the different topics in the field of Virtual Reality applied to Education have been flowing. In this Fig.  15 we can clarify quantitative information such as thematic flow, direction of thematic flow and conversion relationships (Soundararajan et al., 2014 ).

Collaboration between countries mapping

This map gives an improved understanding of the social structure, not only of authors, but also of the countries to which they belong. Figure  16 shows collaborative relationships between China, USA, Korea, and Canada, as well as Germany, Denmark and the United Kingdom, along with others, the collaboration between the USA and China are the most important.

An external file that holds a picture, illustration, etc.
Object name is 10639_2022_11167_Fig16_HTML.jpg

Collaboration between countries

Second stage analysis

This section briefly summarizes the most cited articles, and they are classified into categories, in the literature few authors provide us with a panoramic view of virtual reality technology applied to the educational field. In contrast to other authors (Zappatore et al. ( 2015 ) who perform an eminently quantitative approach in their analyses, this paper follows the line of Heradio et al. ( 2016 ) by providing a dual quantitative–qualitative approach. Thus, our analysis is not limited to counting articles, authors or journals, but describes and comments on the most relevant data for the RV community (Bardakci et al., 2022 ; Kushairi & Ahmi, 2021 ). The articles were listed in descending order according to the number of citations to find the main topics addressed and describe the documents that are considered the most important. All papers which had citations were classified and grouped into six categories: (a) papers about VR-based instruction and learning, (b) papers studying VR learning environments, (c) papers presenting the use of VR in different fields of knowledge, (d) papers describing learning processes that use VR applications, devices or games, (e) papers on research about learning processes using simulation, and (f) topics published during the Covid-19 pandemic.

VR-based instruction and learning

Among the most outstanding papers are the following: Merchant et al. ( 2014 ), Lee et al. ( 2010 ), Makransky and Lilleholt ( 2018 ) and Jensen and Konradsen, ( 2018 ). Jensen and Konradsen, ( 2018 ) seek to update knowledge on the use of head-mounted displays (HMD) in Education and training. The study identifies the acquisition of skills such as cognitive skills, i.e., remembering, understanding information, spatial and visual knowledge; as well as visual exploration or observation, among the most important are the affective skills related to control and emotional response to stressful or difficult situations. These learning tools enhance learning and are very useful in the educational field.

The most cited paper is Merchant et al. ( 2014 ). This work performs a meta-analysis that investigates the effectiveness of reality-based virtual instruction on learning outcomes. In order to do this, the authors researched the overall effect and impact of selected instructional design principles of VR technology-based instruction such as, games, simulation, virtual worlds, in higher education settings and as a result found that using games has a greater effect on learning than simulations and virtual worlds.

The study by Lee et al. ( 2010 ) examined how desktop VR (VR) enhances learning, finding that VR features have an indirect effect on learning. The learning experience was individually measured by psychological factors, such as presence, motivation, cognitive gains, control, and active learning, as well as reflective thinking which all affected the learning outcomes when using the desktop VR-based learning environment. Further research investigated how spatial ability and learning style enable instructional designers and VR software developers to improve learning effectiveness and therefore increase the amount the software is used. According to Makransky and Lilleholt ( 2018 ) much remains to be discovered about the impact and use of immersive VR in e-learning tools that impact students' emotional processes while learning.

Many authors investigate VR as a tool in learning processes. An example is Huang and Liaw, ( 2018 ) who explored how virtual reality technology actively focuses on the learner's interactive learning processes and attempts to reduce the gap between learner knowledge and real-life experience. Alfalah ( 2018 ) examined perceptions when using VR as a tool for education confirming that teachers and students are willing to use VR.

Allcoat and von Mühlenen ( 2018 ) assigned students into three groups who taught with different methods, 1) traditional book learning, 2) virtual reality learning and 3) video. The students were tested for their knowledge of the subject being taught before and after the classes, finding that participants in the virtual reality group showed better recall performance and more positive emotions than the other groups.

Hewawalpita et al. ( 2018 ) explored an improved configuration of massive open online courses. Two groups were used, one group were students who had already taken the traditional course and the second group started from scratch and were given virtual reality content. The results showed that the second group had significantly better performance and it was concluded that interactive learning content can be designed for the different learning needs of students.

Wang, ( 2018 ) proposed a distance learning virtual reality experiment with computers and VR technology and found practical reasons to promote the development of distance learning using computer.

In view of the works analyzed in the context of VR-based Instruction and Learning, we can say that the VR-based learning process is a useful tool for the educator, as it can replicate or complement traditional teaching methods. It is a fully effective concept even using basic forms, such as VR glasses or smartphones. This method is suitable for classroom teaching, distance learning, self-learning and other educational environments, and allows the simulation of scenarios that enrich teaching, even those dangerous experiments that cannot be reproduced in reality.

These papers conclude that it is essential to design useful and learner-friendly VR learning tasks and activities in order to improve learning outcomes. These activities should be adapted to learners with different learning styles and special abilities. Although this is a novel experience, the gap in these research models lies in the need for new longitudinal studies to verify whether improvements in teaching processes are maintained over time. It is necessary to identify differences in teaching processes, such as context, sample, duration, cultural background or learning programs with different content.

There is a need to deepen and explore the influence of virtual reality on the relationship between motivation and learning performance. That is, it is desirable to know whether students' disappointment can have a negative influence on their learning, using qualitative and quantitative methods in the study of prolonged periods.

VR learning environments

Geng et al. ( 2021 ) explore the pedagogical potential of Interactive Spherical Virtual reality based on video in geographic education, considering the perspective of teachers, they were given an introduction to this technology to know the acceptance, creation and experience, it was intended that teachers know the potential of this technology for teaching and learning purposes, the main concerns were the technological integration in pedagogy, they find that they need more professional development to design and refine this methodology. Perhaps it is more change adversity that is reflected in the need to ask for more training in the use of technology.

VR learning environments are explained by Huang et al. ( 2010 ) who indicate that there is a shift in learning from conventional multimedia to a more immersive, interactive, intuitive and exciting VR learning environment. It combines positive pedagogy and the use of technological innovations that are immersive and trigger the imagination of the learner.

Fowler, 2015 tried to give a more pedagogical description of adopting learning in three-dimensional (3-D) virtual learning environments (VLE) using a "design for learning" perspective that is useful for those who design learning activities in 3D VLEs, but considers that the risk of high-fidelity 3D VLEs is that using them to create virtual classrooms that "feel" and look like real classrooms means that they miss the opportunity to create pedagogically new and innovative learning environments.

Yang et al. ( 2010 ) investigated designing and developing a physically interactive learning environment. This was a PILE system that integrated VR video capture technology in a classroom. The group using the system showed a significant difference in pre-test and post-test knowledge. Makransky and Petersen ( 2019 ) believe that VR has the potential to enrich students' educational experiences. The authors investigated the affective and cognitive factors that play a role in learning when using desktop virtual reality simulation and concluded that learners can benefit from desktop virtual reality simulation in which emphasis is given to effective virtual reality features with a high level of usability.

The works analyzed emphasize that previous experiences with virtual reality in education have improved significantly. Although in the beginning they only used a mouse and keyboard as input devices, the benefits and educational effectiveness of 3D virtual learning and new virtual tools such as the PILE system, which allows students to interact with objects on the screen through physical movements, are gradually emerging.

Although technology has accompanied the teaching process exponentially in recent years, replacing traditional whiteboards with smart boards and VR elements, the gap in these research models lies in the need for new research is needed to explore the variables that may affect learning outcomes when using VR simulations. The described works suggest that it would be convenient to explore aspects such as duration, users' prior knowledge or dual cognitive/affective component.

The study of individual student differences, the long-term implications for knowledge acquisition, the frequent use of technology outside of teaching, the ease of use of different VR tools and the willingness of teachers to implement new VR-based utilities are also considered. This analysis evidences the importance of VR-based simulation processes, especially in areas of knowledge development that teachers deem necessary, allowing a balance between the cost and benefit of the experiences obtained.

Use of VR in various fields of knowledge

Osti et al. ( 2021 ) They seek to train construction workers using a novel VR system, this simulated a virtual training site, implemented a 3D training video with a VR head-mounted display, and compared it with a second group shown simple 2-D instructional video training, the first group presented better results in terms of retention, task performance, learning speed and participation. The practical application of VR as a teaching and learning tool is remarkable.

Schmidt and Glaser ( 2021 ) investigated the use of virtual reality by individuals with autism using 360-degree video modeling and headset-based virtual reality to investigate skills acquirement in adults on the autism spectrum in order to promote safety and the appropriate use of public transport. The results suggest a very positive learning experience and that the intervention is feasible and relevant for the unique needs of the target population.

Vélaz et al. ( 2014 ) studied the influence of interaction technology on the learning process when performing assembly tasks and learning processes using games and VR applications for industrial education. Sampaio et al. ( 2013 ) investigated the use of VR in civil engineering education by using it as a tool to create interactive applications as part of research work with students in which VR applications were developed for use in the construction industry.

A study by Eaves et al. ( 2011 ) determined the effects of two variations of real-time VR and feedback when learning a complex dance movement. Crocetta et al. ( 2018 ) presented and described a VR software package that helps in the rehabilitation of people living with disabilities. The findings of the study suggest that motor skills could be influenced differently depending on the environment and interface in which the software is used.

Learning with VR apps, devices or games

Chen and Hsu ( 2020 ) used a VR game-based English mobile learning application to investigate the effectiveness in English learning from a cognitive and psychological perspective, finding that interaction with the virtual reality application and the challenges of a game-based design allow students to enter the flow state easily and enhance their motivation to learn.

Authors M. Zhang et al. ( 2018 ) studied recent developments in game-based VR educational laboratories. According to the author there are several inherent disadvantages of VR that prevent its widespread deployment in the educational field such as unrealistic representation, lack of customization and flexibility, financial feasibility and the physical and psychological discomfort of users.

Sood and Singh ( 2018 ) considered that educational games for electroencephalography (EEG) can be widely used to improve the cognitive and learning skills of students. This can be achieved with the combination of VR and computing that provides accessible e-learning education worldwide. Psotka ( 2013 ) indicated that new technology such as VR and educational games can often disrupt established practices and are therefore considered disruptive technologies. The author believes however that they are appropriate for education and training today but have not been accepted in education due to changing social lifestyles.

Learning processes using simulation

Makransky et al. ( 2020 ) investigated the value of using immersive virtual reality (IVR) laboratory simulations in science education in two studies. The first study used an IVR laboratory safety simulation with pre- and post-test design. The second study compared the value of using IVR simulation and video simulation for learning the topic of DNA analysis. The results show that in both groups there were significant gains in self-efficacy and physical outcome expectations, but the increase in career aspirations and personal outcome expectations did not reach statistical significance.

Hsu et al. ( 2016 ) considered that visual simulation technologies have received considerable attention in learning. A vehicle driving simulation system was created to assist novice drivers in practicing their skills by considering various environmental driving factors that may be encountered while traveling. Dubovi et al. ( 2017 ) evaluated the effectiveness of VR learning simulation in pharmacology for higher education students requiring special skills to learn about medications and the procedure for administering them. The results revealed higher conceptual and procedural knowledge than with solely lecture-based learning.

Topics published during the Covid-19 pandemic

Wu et al. ( 2021 ) They use an immersive virtual reality approach based on video, they developed a landscape architecture VR learning system, due to the fact that during the Covid-19 pandemic the fields are closed, in addition, online education lacks the necessary scenarios for the courses taught during the pandemic, so better results and learning attitudes are achieved than students not subjected to this VR system. The importance of VR as a learning tool is evidenced in the face of the limitation of a real environment, here technology becomes an important ally.

Paszkiewicz et al. ( 2021 ) presented an educational process for Industry 4.0 that included the design, creation, implementation and evaluation of individual courses implemented in a virtual reality environment, identifying significant advantages and disadvantages of VR-based education. The development and implementation of appropriate courses in the virtual reality environment was found to reduce costs and increase the safety and efficiency of activities.

Yerden & Akkuş, ( 2020 ) examined the effects of the use of a Virtual Reality Supported Remote Access Laboratory (VRRALAB) system using remote access and virtual reality technologies on students' learning experience. The interactive use of a real device with a VR-supported remote access laboratory environment does not have any risks for novice users. The results indicate that remote access labs using virtual reality are likely to increase learning quality and student satisfaction levels.

Taçgın, ( 2020 ) investigated the characteristics of an immersive virtual reality learning environment (IVRLE) by evaluating perceived simulation effectiveness for student learning, attitude, and confidence by using gesture interaction to teach preoperative surgical procedures and concepts to undergraduate nursing students. Well-designed and targeted IVRLE was found to help to improve students' confidence in practical skills. Wang, ( 2020 ) applied virtual reality techniques in modular teaching to construct virtual simulation teaching resources and built two teaching modules that are visual, interactive, scalable, upgradable and optimizable. The results of the research suggest a new method of modular teaching and are a useful reference.

The results showed that the production of documents on VR in education has increased since 2015, possibly due to the increase in interest in virtual reality technology. There were important changes in the field in 2014 with the introduction of the Oculus Rift Frame. The interest in VR is reflected in the number of publications in 2018 and has been increasing with the changes in education systems due to the Covid-19 pandemic. The productivity in the last two years has been much higher than before.

The growth is then constant from 2015 onwards, increasing 10 times by 2020. There was a high number of publications in 2019 and a much higher number in 2020, possibly due to the new interest in VR and the impact of the Covid-19 pandemic. Higher production is expected for VR in Education in 2021. The recent growth is consistent with the results of the study on VR and motivation in the educational field (Soto et al., 2020 ) and with the work in the field of rehabilitation (Huang et al., 2016 ).

Figure  14 shows that China leads the number of publications with 273 articles, but interestingly the journals with the highest number of publications are the German International Journal of Emerging Technologies in Learning and Computers & Education from the UK. These journals also had the highest impact in the studied dataset. The difference of China as the leader of publications and the publishing journals may be due to the papers contained in the selection process. From the total of 298 journals and 718 papers that make up the dataset, there are 72 papers that were written by a single author. Most publications were written by an average of 2.95 authors, which means that from 1939 authors there are 1867 authors with papers with multiple authors. These papers are widely cited, with an average number of 10.03 citations per paper over a 10-year period and an annual average number of 2.49 citations per paper. Interestingly, most citations were in 2010 with an average annual number of 6.97 citations per paper, 2014 with 7.13 citations per paper and 2018 with 5.99. The data for the Covid-19 pandemic period, which started in 2020, is clearly shown in Fig.  4 . The two most cited papers were (Merchant et al., 2014 ) with 574 citations in volume 70 of the Computers & Education journal and (Huang et al., 2010 ) with 321 citations in volume 55 of the same journal.

Figure  7 presents the authors with the highest impact after analyzing values of the h-index. In this study they were found to be Chen Y. and Jong M. both with an h-index value of 7. Figure  6 shows the authors Chen W., Chen Y. and Lee J., who are considered very productive because they have 8 publications.

Co-occurrence was used to identify the conceptual structure by analyzing the words in the Keywords Plus of cited articles. The words that appear most often were found to be virtual reality with 107 occurrences, students with 70, education with 66, learning with 58 occurrences, and environments with 44. All these words are closely related to the search topics although technology, performance, simulation, design and learning systems were also found. This information is shown in Fig.  10 .

The words covering the topics in the Keywords Plus of the documents were analyzed by means of a factorial map and a cluster analysis, as shown in Fig.  9 . Two dimensions were used, the first of which covers topics related to VR visualization devices by users of virtual reality educational products, and the second dimension covers VR uses in education. Words related to visualization, attention and motivation of students, the use of serious games, interactive learning environments, online learning and user experience to strengthen the use of VR in education were all found in this analysis stage.

Co-citation was used to analyze the intellectual structure. The authors of the documents were analyzed, and Lee was found to have a co-citation relationship Merchant, Dalgarno, Smith, Mikropoulos, Davis and Bouman, while a second group of co-citation authors was formed by Chen, Wang, Huang, Zhang, Chang, Yang and Lin. This information is shown in Fig.  11 .

An inductive analysis found that the most cited papers were into the major categories of virtual reality-based instruction and learning, learning environments using VR, VR for teaching learning processes, instructional design and VR as a tool that enhances learning processes.

VR learning environments contain different systems where VR is applied in education, ranging from conventional multimedia to a physically interactive learning environment.

Research was found that investigated the use of VR in different fields of knowledge and various areas of education such as civil engineering, production lines and rehabilitation.

Learning processes incorporating the use of VR applications, devices or games are considered disruptive technologies that alter the usual teaching, cognitive and learning practices of students. However, the combination of VR with computing was seen to provide effective e-learning.

Articles were also found in the dataset about research using simulation models, for example driving vehicles, assembly lines, operations in the medical field. These simulation models used three-dimensional technology for the teaching process.

The research topics published during the Covid-19 pandemic examined the new trend in the field of education caused by the abrupt change from face-to-face teaching to virtual, online teaching. Research topics included deep learning and machine learning using artificial intelligence as alternatives to everyday teaching technologies and platforms. These included robot teleoperation, remote access laboratories, Virtual Reality-Based Cognitive Telerehabilitation Systems, Machine Learning Predictions, Immersive Virtual Reality learning environments, intelligent virtual reality technology, distance learning classrooms using machine learning and virtual reality, Virtual Reality-Interactive Classroom, Student Orientation in Distance Education Programs, Virtual Reality and BIM Methodology for Teaching–Learning Improvement, virtual reality-based gaming instruction and virtual reality for students' adaptive learning among others.

The six categories of the second analysis showed that improvements in VR learning processes have occurred in recent years and important advances have been made in the application and use of this technology. However, even though the engagement and motivation of students can be improved by using this technology, there still remains a lot to be discovered about the use of e-learning tools. VR learning environments have advanced significantly from multimedia to interactive environments with desktop and immersive VR processes, 360-degree tours and various applications and games. Education has generally adopted these changes slowly with most attention coming from the new generations of students. VR technology has existed for many years but has not been adopted as quickly as would be expected by educational centers, where it is seen as a disruptive process and is more commonly used by students for recreational purposes. The results of this study show that despite the multiple applications of VR in different fields of education, there is little evidence of incorporating them in teaching–learning processes. Educators have, however, quickly resorted to the use of technology platforms to teach because of the sudden change from face-to-face to virtual, online classes due to the Covid-19 pandemic. With the many available applications of VR in education and given that students have accepted the use of VR for recreational purposes, it can be incorporated as part of educational and training courses and demonstrations, as well as being useful for evaluating students.

Finally, the quality assessment of this research work was evaluated applying contrasted methods, following recommendations proposed by Ramey and Rao ( 2011 ) and Rao and Ramey ( 2011 ). Similarly, the quality of the research work was evaluated by incorporating bibliometric criteria, especially in the automation of the whole process, by Pulsiri and Vatananan-Thesenvitz ( 2018 ). External validity was contrasted with the criteria of dos Santos Rocha and Fantinato ( 2013 ) and Nguyen-Duc et al. ( 2015 ).

Conclusions

Bibliometric analysis is a valuable tool that can be used to reveal the evolution of the articles contained in the dataset and answer research questions.

The conceptual structure shows that the most-used words are related to the search terms, although some words were found to be used with higher frequency. These were virtual reality, students, education, e learning, teaching, while the main topics investigated range from basic ones such as virtual reality and education to technology, teaching, visualization, student motivation and attention, e-learning, learning system, deep learning, and immersive virtual reality.

Recent publications show an increasing interest in VR, but rather than delving deeper into this technology other technological options are being explored with combinations of VR and other technology and systems. The pandemic improved our abilities to adapt and innovate with virtuality, along with teleworking and virtual teams (Garro-Abarca et al., 2020 , 2021 ). There has been a lot of research into tools that allow educators to improve and even reinvent teaching processes. Virtual reality applications are interactive and immersive with telepresence and education might reconsider its opinion of this technology and take advantage of it to make classes more enjoyable with the new normality we are living.

One of the contributions of this study has been to confirm the progress of VR technology. In the future, educational centers will be able to solve many challenges, such as learning by experiencing and interacting with an environment, instead of passively receiving the information to be assimilated. VR favors the motivation and involvement of students and educational staff in general, in addition to increasing the speed of learning. b. The frontier and future of VR learning environments. On the other hand, the frontier and future of VR learning environments, the article predicts that the increasing diffusion of virtual reality learning environments forces to create limits. One of the limits is based on the capacity of these didactic tools to improve their effectiveness at a training level compared to other more traditional methods. In addition, other aspects such as user privacy and equal opportunities for all students must be taken into consideration.

Thus, the trends of type and use of VR in different fields of knowledge will be directed towards the inclusion of experiential VR models, adapted to each field of knowledge. The aim is to create learning vehicles that enhance the acquisition of knowledge by students in specific fields of knowledge. In this context, the training of teachers in the use of these technologies within a clear educational framework that is adaptable to the different academic disciplines will be increasingly important. In this sense, the future of VR in education will depend to a large extent on the motivation generated in students, and on being able to identify the necessary characteristics to achieve an adequate level of learning through VR. The push for VR-based education, brought about by the Covid-19 pandemic, is expected to make this style of learning one of the educational preferences in the future, especially for self-learning and teaching in certain areas where VR technology is more developed.

This study was limited to investigating the use of virtual reality technology in education. Articles dealing with the medical field were not selected as primary data sources because the focus of research was on the application of VR in teaching and learning processes in secondary and university education, as well as other applications in conferences or training processes. However, articles were included from multidisciplinary areas that include research in the medical field to broaden the spectrum of practical applications of this technology.

Suggestions for future bibliometric analyses include the evolution of other subject areas, using new search terms that allow other articles related to the field of education to be included for a broader analysis of the metadata and investigating other present and future lines of research.

Acknowledgements

Not applicable.

Authors' contributions

All authors read and approved the final manuscript.

Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature.

Data availability

Declarations.

The authors declare that they have no conflicting interests.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

  • Alfalah SF. Perceptions toward adopting virtual reality as a teaching aid in information technology. Education and Information Technologies. 2018; 23 (6):2633–2653. doi: 10.1007/s10639-018-9734-2. [ CrossRef ] [ Google Scholar ]
  • Allcoat, D., & von Mühlenen, A. (2018). Learning in virtual reality: Effects on performance, emotion and engagement. Research in Learning Technology , 26 . 10.25304/rlt.v26.2140
  • Altınay Ozdemir MA, Goktas LS. Research trends on digital detox holiday: A bibliometric analysis, 2012–2020. Tourism & Management Studies. 2021; 17 (3):21–35. doi: 10.18089/tms.2021.170302. [ CrossRef ] [ Google Scholar ]
  • Álvarez-García J, Maldonado-Erazo CP, Río-Rama D, Cruz M, Castellano-Álvarez FJ. Cultural Heritage and Tourism Basis for Regional Development: Mapping of Scientific Coverage. Sustainability. 2019; 11 (21):6034. doi: 10.3390/su11216034. [ CrossRef ] [ Google Scholar ]
  • Aria M, Cuccurullo C. bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics. 2017; 11 (4):959–975. doi: 10.1016/j.joi.2017.08.007. [ CrossRef ] [ Google Scholar ]
  • Bardakci S, Soylu MY, Akkoyunlu B, et al. Collaborations, concepts, and citations in educational technology: A trend study via bibliographic mapping. Education and Information Technologies. 2022; 27 :4321–4346. doi: 10.1007/s10639-021-10785-9. [ CrossRef ] [ Google Scholar ]
  • Beyer-Berjot L, Berdah S, Hashimoto DA, Darzi A, Aggarwal R. A virtual reality training curriculum for laparoscopic colorectal surgery. Journal of Surgical Education. 2016; 73 (6):932–941. doi: 10.1016/j.jsurg.2016.05.012. [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Blanco-Mesa F, Merigó JM, Gil-Lafuente AM. Fuzzy decision making: A bibliometric-based review. Journal of Intelligent & Fuzzy Systems. 2017; 32 (3):2033–2050. doi: 10.3233/JIFS-161640. [ CrossRef ] [ Google Scholar ]
  • Bonetti, F., Warnaby, G., & Quinn, L. (2018). Augmented reality and virtual reality in physical and online retailing: A review, synthesis and research agenda. In Augmented reality and virtual reality (pp. 119–132). Springer. 10.1007/978-3-319-64027-3_9
  • Börner K, Chen C, Boyack KW. Visualizing knowledge domains. Annual Review of Information Science and Technology. 2003; 37 (1):179–255. doi: 10.1002/aris.1440370106. [ CrossRef ] [ Google Scholar ]
  • Brey, P. (2014). Virtual reality and computer simulation. In Ethics and emerging technologies (pp. 315–332). Palgrave Macmillan. 10.1057/9781137349088_21
  • Buehl D. Classroom strategies for interactive learning. Stenhouse Publishers; 2017. [ Google Scholar ]
  • Calabuig-Moreno F, González-Serrano MH, Fombona J, García-Tascón M. The emergence of technology in physical education: A general bibliometric analysis with a focus on virtual and augmented reality. Sustainability. 2020; 12 (7):2728. doi: 10.3390/su12072728. [ CrossRef ] [ Google Scholar ]
  • Castillo HGC. The triple helix model as a means for university-business linkage. National Journal of Administration. 2010; 1 (1):85–94. doi: 10.22458/rna.v1i1.286. [ CrossRef ] [ Google Scholar ]
  • Chen YL, Hsu CC. Self-regulated mobile game-based English learning in a virtual reality environment. Computers & Education. 2020; 154 :103910. doi: 10.1016/j.compedu.2020.103910. [ CrossRef ] [ Google Scholar ]
  • Cheng F-F, Huang Y-W, Yu H-C, Wu C-S. Mapping knowledge structure by keyword co-occurrence and social network analysis. Library Hi Tech. 2018; 36 (4):636–650. doi: 10.1108/lht-01-2018-0004. [ CrossRef ] [ Google Scholar ]
  • Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F. Science mapping software tools: Review, analysis, and cooperative study among tools. Journal of the American Society for Information Science and Technology. 2011; 62 (7):1382–1402. doi: 10.1002/asi.21525. [ CrossRef ] [ Google Scholar ]
  • Crocetta TB, Araújo LV, Guarnieri R, Massetti T, Ferreira FHIB, De Abreu LC, Mello Monteiro CB. Virtual reality software package for implementing motor learning and rehabilitation experiments. Virtual Reality. 2018; 22 (3):199–209. doi: 10.1007/s10055-017-0323-2. [ CrossRef ] [ Google Scholar ]
  • Danvila-del-Valle I, Estévez-Mendoza C, Lara FJ. Human resources training: A bibliometric analysis. Journal of Business Research. 2019; 101 :627–636. doi: 10.1016/j.jbusres.2019.02.026. [ CrossRef ] [ Google Scholar ]
  • Di Natale AF, Repetto C, Riva G, Villani D. Immersive virtual reality in K-12 and higher education: A 10-year systematic review of empirical research. British Journal of Educational Technology. 2020; 51 (6):2006–2033. doi: 10.1111/bjet.13030. [ CrossRef ] [ Google Scholar ]
  • dos Santos Rocha R, Fantinato M. The use of software product lines for business process management: A systematic literature review. Information and Software Technology. 2013; 55 (8):1355–1373. doi: 10.1016/j.infsof.2013.02.007. [ CrossRef ] [ Google Scholar ]
  • Dubovi I, Levy ST, Dagan E. Now I know how! The learning process of medication administration among nursing students with non-immersive desktop virtual reality simulation. Computers & Education. 2017; 113 :16–27. doi: 10.1016/j.compedu.2017.05.009. [ CrossRef ] [ Google Scholar ]
  • Durán Sánchez, A., Álvarez-García, J., Río-Rama, D., & Cruz, M. (2014). Active tourism research: A literature review. ROTUR, 8 , 62–76.
  • Durán-Sánchez A, Álvarez-García J, Río-Rama D, Cruz M, Oliveira C. Religious tourism and pilgrimage: Bibliometric overview. Religions. 2018; 9 (9):249. doi: 10.3390/rel9090249. [ CrossRef ] [ Google Scholar ]
  • Durán-Sánchez A, Álvarez-García J, Río-Rama MDL, Rosado- Cebrián B. Science mapping of the knowledge base on tourism innovation. Sustainability. 2019; 11 (12):3352. doi: 10.3390/su11123352. [ CrossRef ] [ Google Scholar ]
  • Eaves DL, Breslin G, Van Schaik P, Robinson E, Spears IR. The short-term effects of real-time virtual reality feedback on motor learning in dance. Presence: Teleoperators and Virtual Environments. 2011; 20 (1):62–77. doi: 10.1162/pres_a_00035. [ CrossRef ] [ Google Scholar ]
  • Fagan M, Kilmon C, Pandey V. Exploring the adoption of a virtual reality simulation: The role of perceived ease of use, perceived usefulness and personal innovativeness. Campus-Wide Information Systems. 2012; 29 (2):117–127. doi: 10.1108/10650741211212368. [ CrossRef ] [ Google Scholar ]
  • Falagas ME, Pitsouni EI, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, web of science, and Google scholar: Strengths and weaknesses. The FASEB Journal. 2008; 22 (2):338–342. doi: 10.1096/fj.07-9492lsf. [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Folgado-Fernández JA, Palos-Sánchez PR, Camacho MA. Motivaciones, formación y planificación del trabajo en equipo para entornos de aprendizaje virtual. Interciencia. 2020; 45 (2):102–109. [ Google Scholar ]
  • Fowler C. Virtual reality and learning: Where is the pedagogy? British Journal of Educational Technology. 2015; 46 (2):412–422. doi: 10.1111/bjet.12135. [ CrossRef ] [ Google Scholar ]
  • Freina, L., Bottino, R., & Tavella, M. (2016). From e-learning to VR-learning: An example of learning in an immersive virtual world. Journal of E-Learning and Knowledge Society , 12 (2). https://www.learntechlib.org/p/173465/
  • Garro-Abarca V, Palos-Sanchez P, Aguayo-Camacho M. Virtual teams in times of pandemic: Factors that influence performance. Frontiers in Psychology. 2021; 12 :232. doi: 10.3389/fpsyg.2021.624637. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Garro-Abarca VM, Palos-Sanchez PR, Rus-Arias E. Working in virtual teams: A systematic literature review and a bibliometric analysis. IEEE Access. 2020; 8 :168923–168940. doi: 10.1109/ACCESS.2020.3023546. [ CrossRef ] [ Google Scholar ]
  • Geng J, Chai CS, Jong MSY, Luk ETH. Understanding the pedagogical potential of Interactive Spherical Video-based Virtual Reality from the teachers’ perspective through the ACE framework. Interactive Learning Environments. 2021; 29 (4):618–633. doi: 10.1080/10494820.2019.1593200. [ CrossRef ] [ Google Scholar ]
  • Gokhale A, Mulay P, Pramod D, Kulkarni R. A bibliometric analysis of digital image forensics. Science & Technology Libraries. 2020; 1 :18. doi: 10.1080/0194262x.2020.1714529. [ CrossRef ] [ Google Scholar ]
  • Gómez-García, M., Trujillo-Torres, J. M., Aznar-Díaz, I., & Cáceres-Reche, M. P. (2018). Augment reality and virtual reality for the improvement of spatial competences in Physical Education . 10.14198/jhse.2018.13.proc2.03
  • Guo Y-M, Huang Z-L, Guo J, Li H, Guo X-R, Nkeli MJ. Bibliometric analysis on smart cities research. Sustainability. 2019; 11 (13):3606. doi: 10.3390/su11133606. [ CrossRef ] [ Google Scholar ]
  • Guttentag DA. Virtual reality: Applications and implications for tourism. Tourism Management. 2010; 31 (5):637–651. doi: 10.1016/j.tourman.2009.07.003. [ CrossRef ] [ Google Scholar ]
  • Hassan SU, Visvizi A, Waheed H. The ‘who’and the ‘what’in international migration research: Data-driven analysis of Scopus-indexed scientific literature. Behaviour & Information Technology. 2019; 38 (9):924–939. doi: 10.1080/0144929x.2019.1583282. [ CrossRef ] [ Google Scholar ]
  • Hen, L. B. (2019). Exploring Surgeon’s Acceptance of Virtual Reality Headset for Training. In Augmented Reality and Virtual Reality (pp. 291–304). Springer. 10.1007/978-3-030-06246-0_21
  • Heradio R, De La Torre L, Galan D, Cabrerizo FJ, Herrera-Viedma E, Dormido S. Virtual and remote labs in education: A bibliometric analysis. Computers & Education. 2016; 98 :14–38. doi: 10.1016/j.compedu.2016.03.010. [ CrossRef ] [ Google Scholar ]
  • Hernández-Torrano, D., & Kuzhabekova, A. (2019). The state and development of research in the field of gifted education over 60 years: A bibliometric study of four gifted education journals (1957–2017). High Ability Studies , 1–23. 10.1080/13598139.2019.1601071
  • Hewawalpita S, Herath S, Perera I, Meedeniya D. effective learning content offering in MOOCs with virtual reality-an exploratory study on learner experience. Journal of Universal Computer Science. 2018; 24 (2):129–148. [ Google Scholar ]
  • Hsu, K. S., Jiang, J. F., Wei, H. Y., & Lee, T. H. (2016). Application of the environmental sensation learning vehicle simulation platform in virtual reality. Eurasia Journal of Mathematics, Science & Technology Education , 12 (5). 10.12973/eurasia.2016.1525a
  • Huang, H. M., & Liaw, S. S. (2018). An analysis of learners’ intentions toward virtual reality learning based on constructivist and technology acceptance approaches. International Review of Research in Open and Distributed Learning , 19 (1). 10.19173/irrodl.v19i1.2503
  • Huang HM, Rauch U, Liaw SS. Investigating learners’ attitudes toward virtual reality learning environments: Based on a constructivist approach. Computers & Education. 2010; 55 (3):1171–1182. doi: 10.1016/j.compedu.2010.05.014. [ CrossRef ] [ Google Scholar ]
  • Huang Y, Huang Q, Ali S, Zhai X, Bi X, Liu R. Rehabilitation using virtual reality technology: A bibliometric analysis, 1996–2015. Scientometrics. 2016; 109 (3):1547–1559. doi: 10.1007/s11192-016-2117-9. [ CrossRef ] [ Google Scholar ]
  • Huber JC. A new model that generates Lotka’s law. Journal of the American Society for Information Science and Technology. 2002; 53 (3):209–219. doi: 10.1002/asi.10025. [ CrossRef ] [ Google Scholar ]
  • Hussin, N. H., Jaafar, J., & Downe, A. G. (2011). Assessing educators’ acceptance of Virtual Reality (VR) in the classroom using the Unified Theory of Acceptance and Use of Technology (UTAUT. International Visual Informatics Conference , 216–225. 10.1007/978-3-642-25191-7_21
  • Huttar CM, BrintzenhofeSzoc K. Virtual reality and computer simulation in social work education: A systematic review. Journal of Social Work Education. 2020; 56 (1):131–141. doi: 10.1080/10437797.2019.1648221. [ CrossRef ] [ Google Scholar ]
  • Hwang WY, Hu SS. Analysis of peer learning behaviors using multiple representations in virtual reality and their impacts on geometry problem solving. Computers & Education. 2013; 62 :308–319. doi: 10.1016/j.compedu.2012.10.005. [ CrossRef ] [ Google Scholar ]
  • Jensen L, Konradsen F. A review of the use of virtual reality head-mounted displays in education and training. Education and Information Technologies. 2018; 23 (4):1515–1529. doi: 10.1007/s10639-017-9676-0. [ CrossRef ] [ Google Scholar ]
  • Johnson-Glenberg MC. Immersive VR and education: Embodied design principles that include gesture and hand controls. Frontiers in Robotics and AI. 2018; 5 :81. doi: 10.3389/frobt.2018.00081. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Kamińska D, Sapiński T, Wiak S, Tikk T, Haamer RE, Avots E, Anbarjafari G. Virtual reality and its applications in education: Survey. Information. 2019; 10 (10):318. doi: 10.3390/info10100318. [ CrossRef ] [ Google Scholar ]
  • Kartiko I, Kavakli M, Cheng K. Learning science in a virtual reality application: The impacts of animated-virtual actors’ visual complexity. Computers & Education. 2010; 55 (2):881–891. doi: 10.1016/j.compedu.2010.03.019. [ CrossRef ] [ Google Scholar ]
  • Katsioloudis, P., Jones, M., & Jovanovic, V. (2017). Use of virtual reality head-mounted displays for engineering technology students and implications on spatial visualization ability as measured through rotational view drawings. The Engineering Design Graphics Journal , 81 (1). Winter
  • Kushairi, N., & Ahmi, A. (2021). Flipped classroom in the second decade of the Millenia: A bibliometrics analysis with Lotka’s law. Education and Information Technologies , 26, 4401–4431. 10.1007/s10639-021-10457-8 [ PMC free article ] [ PubMed ]
  • Kwon C. Verification of the possibility and effectiveness of experiential learning using HMD-based immersive VR technologies. Virtual Reality. 2019; 23 (1):101–118. doi: 10.1007/s10055-018-0364-1. [ CrossRef ] [ Google Scholar ]
  • Lampert B, Pongracz A, Sipos J, Vehrer A, Horvath I. MaxWhere VR-learning improves effectiveness over clasiccal tools of e-learning. Acta Polytechnica Hungarica. 2018; 15 (3):125–147. doi: 10.12700/aph.15.3.2018.3.8. [ CrossRef ] [ Google Scholar ]
  • Lazar I, Panisoara IO. Understanding the role of modern technologies in education: A scoping review protocol. Psychreg J. Psychol. 2018; 2 :74–86. [ Google Scholar ]
  • Le QT, Pedro A, Park CS. A social virtual reality based construction safety education system for experiential learning. Journal of Intelligent & Robotic Systems. 2015; 79 (3–4):487–506. doi: 10.1007/s10846-014-0112-z. [ CrossRef ] [ Google Scholar ]
  • Lee EAL, Wong KW, Fung CC. How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach. Computers & Education. 2010; 55 (4):1424–1442. doi: 10.1016/j.compedu.2010.06.006. [ CrossRef ] [ Google Scholar ]
  • Lövquist E, Shorten G, Aboulafia A. Virtual reality-based medical training and assessment: The multidisciplinary relationship between clinicians, educators and developers. Medical Teacher. 2012; 34 (1):59–64. doi: 10.3109/0142159x.2011.600359. [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lv Z, Li X, Li W. Virtual reality geographical interactive scene semantics research for immersive geography learning. Neurocomputing. 2017; 254 :71–78. doi: 10.1016/j.neucom.2016.07.078. [ CrossRef ] [ Google Scholar ]
  • Maas MJ, Hughes JM. Virtual, augmented and mixed reality in K–12 education: A review of the literature. Technology, Pedagogy and Education. 2020; 29 (2):231–249. doi: 10.1080/1475939x.2020.1737210. [ CrossRef ] [ Google Scholar ]
  • Makransky G, Lilleholt L. A structural equation modeling investigation of the emotional value of immersive virtual reality in education. Educational Technology Research and Development. 2018; 66 (5):1141–1164. doi: 10.1007/s11423-018-9581-2. [ CrossRef ] [ Google Scholar ]
  • Makransky G, Petersen GB. Investigating the process of learning with desktop virtual reality: A structural equation modeling approach. Computers & Education. 2019; 134 :15–30. doi: 10.1016/j.compedu.2019.02.002. [ CrossRef ] [ Google Scholar ]
  • Makransky G, Petersen GB, Klingenberg S. Can an immersive virtual reality simulation increase students’ interest and career aspirations in science? British Journal of Educational Technology. 2020; 51 (6):2079–2097. doi: 10.1111/bjet.12954. [ CrossRef ] [ Google Scholar ]
  • Mann, M. K., Liu-Thompkins, Y., Watson, G. S., & Papelis, Y. E. (2015). A multidisciplinary examination of 3D virtual shopping environments: Effects on consumer perceptual and physiological responses. In Ideas in marketing: Finding the new and polishing the old (pp. 752–755). Springer. 10.1007/978-3-319-10951-0_277
  • Martínez-López FJ, Merigó JM, Valenzuela-Fernández L, Nicolás C. Fifty years of the european journal of marketing: A bibliometric analysis. European Journal of Marketing. 2018; 52 (1/2):439–468. doi: 10.1108/EJM-11-2017-0853. [ CrossRef ] [ Google Scholar ]
  • Mas-Tur A, Modak NM, Merigó JM, Roig-Tierno N, Geraci M, Capecchi V. Half a century of quality & quantity: A bibliometric review. Quality & Quantity. 2019; 53 (2):981–1020. doi: 10.1007/s11135-018-0799-1. [ CrossRef ] [ Google Scholar ]
  • McGovern E, Moreira G, Luna-Nevarez C. An application of virtual reality in education: Can this technology enhance the quality of students’ learning experience? Journal of Education for Business. 2019; 95 (7):490–496. doi: 10.1080/08832323.2019.1703096. [ CrossRef ] [ Google Scholar ]
  • Merchant, Z., Goetz, E. T., Keeney-Kennicutt, W., Kwok, O. M., Cifuentes, L., & Davis, T. J. (2012). The learner characteristics, features of desktop 3D virtual reality environments, and college chemistry instruction: A structural equation modeling analysis. Computers & Education, 59 (2), 551–568. 10.1016/j.compedu.2012.02.004
  • Merchant Z, Goetz ET, Cifuentes L, Keeney-Kennicutt W, Davis TJ. Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta- analysis. Computers & Education. 2014; 70 :29–40. doi: 10.1016/j.compedu.2013.07.033. [ CrossRef ] [ Google Scholar ]
  • Mihelj, M., Novak, D., & Beguš, S. (2014). Virtual reality technology and applications . 10.1007/978-94-007-6910-6
  • Milian EZ, Spinola MDM, Carvalho MM. Fintechs: A literature review and research agenda. Electronic Commerce Research and Applications. 2019; 34 :100833. doi: 10.1016/j.elerap.2019.100833. [ CrossRef ] [ Google Scholar ]
  • Moro, C., Štromberga, Z., & Stirling, A. (2017). Virtualisation devices for student learning: Comparison between desktop-based (Oculus Rift) and mobile-based (Gear VR) virtual reality in medical and health science education. Australasian Journal of Educational Technology, 33 (6). 10.14742/ajet.3840
  • Mystakidis S, Berki E, Valtanen JP. Deep and meaningful e-learning with social virtual reality environments in higher education: A systematic literature review. Applied Sciences. 2021; 11 (5):2412. doi: 10.3390/app11052412. [ CrossRef ] [ Google Scholar ]
  • Nguyen-Duc A, Cruzes DS, Conradi R. The impact of global dispersion on coordination, team performance and software quality–a systematic literature review. Information and Software Technology. 2015; 57 :277–294. doi: 10.1016/j.infsof.2014.06.002. [ CrossRef ] [ Google Scholar ]
  • Osti F, de Amicis R, Sanchez CA, Tilt AB, Prather E, Liverani A. A VR training system for learning and skills development for construction workers. Virtual Reality. 2021; 25 (2):523–538. doi: 10.1007/s10055-020-00470-6. [ CrossRef ] [ Google Scholar ]
  • Palos-Sanchez, P. R., Saura, J. R., & Debasa, F. (2018). The influence of social networks on the development of recruitment actions that favor user interface design and conversions in mobile applications powered by linked data. Mobile Information Systems, 2018.
  • Palos-Sanchez, P. R., Folgado-Fernandez, J. A., & Rojas-Sanchez, M. (2022). Virtual reality technology: Analysis based on text and opinion mining. Mathematical Biosciences and Engineering, 19 (8), 7856–7885. 10.3934/mbe.2022367 [ PubMed ]
  • Paszkiewicz A, Salach M, Dymora P, Bolanowski M, Budzik G, Kubiak P. Methodology of implementing virtual reality in education for industry 4.0. Sustainability. 2021; 13 (9):5049. doi: 10.3390/su13095049. [ CrossRef ] [ Google Scholar ]
  • Pham HC, Dao N, Pedro A, Le QT, Hussain R, Cho S, Park CSK. Virtual field trip for mobile construction safety education using 360-degree panoramic virtual reality. International Journal of Engineering Education. 2018; 34 :1174–1191. [ Google Scholar ]
  • Pritchard A. Statistical bibliography or bibliometrics. Journal of Documentation. 1969; 25 (4):348–349. doi: 10.1108/eb026482. [ CrossRef ] [ Google Scholar ]
  • Psotka, J. (2013). Educational games and virtual reality as disruptive technologies.   Educational Technology and Society 16 (2), 69–80 https://www.jstor.org/stable/jeductechsoci.16.2.69
  • Pulsiri, N., & Vatananan-Thesenvitz, R. (2018). Improving systematic literature review with automation and bibliometrics. In 2018 Portland International Conference on Management of Engineering and Technology (PICMET) (pp. 1–8). IEEE.
  • Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I. A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education. 2020; 147 :103778. doi: 10.1016/j.compedu.2019.103778. [ CrossRef ] [ Google Scholar ]
  • Ramey, J., & Rao, P. G. (2011). The systematic literature review as a research genre. In 2011 IEEE International Professional Communication Conference (pp. 1–7). IEEE.
  • Rao, P. G., & Ramey, J. (2011). Use of mobile phones by non-literate and semi-literate people: A systematic literature review. In 2011 IEEE International Professional Communication Conference (pp. 1–10). IEEE.
  • Rodríguez ARR, Navarro JR. Intellectual basis of entrepreneurship research: A biometric study. European Journal of Management and Business Economics. 2008; 17 (1):13–38. [ Google Scholar ]
  • Roemer RC, Borchardt R. Meaningful metrics: A 21st century librarian’s guide to bibliometrics, altmetrics, and research impact. Association of College and Research Libraries; 2015. [ Google Scholar ]
  • Román-Ibáñez V, Pujol-López FA, Mora-Mora H, Pertegal-Felices ML, Jimeno-Morenilla A. A low-cost immersive virtual reality system for teaching robotic manipulators programming. Sustainability. 2018; 10 (4):1102. doi: 10.3390/su10041102. [ CrossRef ] [ Google Scholar ]
  • Sampaio A, Rosario D, Gomes A, Santos J. Virtual reality applied on civil engineering education: Construction activity supported on interactive models. International Journal of Engineering Education. 2013; 29 (6):1331–1347. [ Google Scholar ]
  • Schmidt, M., & Glaser, N. (2021). Investigating the usability and learner experience of a virtual reality adaptive skills intervention for adults with autism spectrum disorder. Educational Technology Research and Development , 1–35. 10.1007/s11423-021-10005-8
  • Shen CW, Ho JT, Ly PTM, Kuo TC. Behavioural intentions of using virtual reality in learning: Perspectives of acceptance of information technology and learning style. Virtual Reality. 2019; 23 (3):313–324. doi: 10.1007/s10055-018-0348-1. [ CrossRef ] [ Google Scholar ]
  • Shen H, Zhang J, Yang B, Jia B. Development of an educational virtual reality training system for marine engineers. Computer Applications in Engineering Education. 2019; 27 (3):580–602. doi: 10.1002/cae.22099. [ CrossRef ] [ Google Scholar ]
  • Small H. Visualizing science by citation mapping. Journal of the American Society for Information Science. 1999; 50 (9):799–813. doi: 10.1002/(sici)1097-4571(1999)50:9. [ CrossRef ] [ Google Scholar ]
  • Sood SK, Singh KD. An Optical-Fog assisted EEG-based virtual reality framework for enhancing E-learning through educational games. Computer Applications in Engineering Education. 2018; 26 (5):1565–1576. doi: 10.1002/cae.21965. [ CrossRef ] [ Google Scholar ]
  • Soto, M. N. C., Navas-Parejo, M. R., & Guerrero, A. J. M. (2020). Virtual reality and motivation in the educational context: Bibliometric study of the last twenty years from Scopus . 10.17163/alt.v15n1.2020.04
  • Soundararajan K, Ho H, Su B. Sankey diagram framework for energy and exergy flows. Applied Energy. 2014; 136 :1035–1042. doi: 10.1016/j.apenergy.2014.08.070. [ CrossRef ] [ Google Scholar ]
  • Srivastava P, Rimzhim A, Vijay P, Singh S, Chandra S. Desktop VR is better than non-ambulatory HMD VR for spatial learning. Frontiers in Robotics and AI. 2019; 6 :50. doi: 10.3389/frobt.2019.00050. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Taçgın Z. The perceived effectiveness regarding Immersive Virtual Reality learning environments changes by the prior knowledge of learners. Education and Information Technologies. 2020; 25 (4):2791–2809. doi: 10.1007/s10639-019-10088-0. [ CrossRef ] [ Google Scholar ]
  • Tussyadiah IP, Jung TH, Dieck MC. Embodiment of wearable augmented reality technology in tourism experiences. Journal of Travel Research. 2018; 57 (5):597–611. doi: 10.1177/0047287517709090. [ CrossRef ] [ Google Scholar ]
  • Tzanavari A, Tsapatsoulis N. Affective, interactive, and cognitive methods for e-learning design: Creating an optimal education experience. Information Science Reference. 2010 doi: 10.4018/978-1-60566-940-3. [ CrossRef ] [ Google Scholar ]
  • Van Eck NJ, Waltman L. Bibliometric mapping of the computational intelligence field. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems. 2007; 15 (05):625–645. doi: 10.1142/s0218488507004911. [ CrossRef ] [ Google Scholar ]
  • Velasco CAB, Parra VFG, García CQ. Evolution of the literature on family business as a scientific discipline. Cuadernos De Economía y Dirección De La Empresa. 2011; 14 (2):78–90. doi: 10.1016/j.cede.2011.02.004. [ CrossRef ] [ Google Scholar ]
  • Vélaz, Y., Rodríguez Arce, J., Gutiérrez, T., Lozano-Rodero, A., & Suescun, A. (2014). The influence of interaction technology on the learning of assembly tasks using virtual reality. Journal of Computing and Information Science in Engineering , 14 (4). 10.1115/1.4028588
  • Vignais N, Kulpa R, Brault S, Presse D, Bideau B. Which technology to investigate visual perception in sport: Video vs Virtual Reality. Human Movement Science. 2015; 39 :12–26. doi: 10.1016/j.humov.2014.10.006. [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Wang F. Computer distance virtual experiment teaching application based on virtual reality technology. International Journal of Emerging Technologies in Learning (IJET) 2018; 13 (04):83–94. doi: 10.3991/ijet.v13i04.8472. [ CrossRef ] [ Google Scholar ]
  • Wang Y. Application of virtual reality technique in the construction of modular teaching resources. International Journal of Emerging Technologies in Learning (IJET) 2020; 15 (10):126–139. doi: 10.3991/ijet.v15i10.14129. [ CrossRef ] [ Google Scholar ]
  • Wu WL, Hsu Y, Yang QF, Chen JJ. A spherical video-based immersive virtual reality learning system to support landscape architecture students’ learning performance during the COVID-19 era. Land. 2021; 10 (6):561. doi: 10.3390/land10060561. [ CrossRef ] [ Google Scholar ]
  • Xie H, Zhang Y, Wu Z, Lv T. A bibliometric analysis on land degradation: Current status, development, and future directions. Land. 2020; 9 (1):28. doi: 10.3390/land9010028. [ CrossRef ] [ Google Scholar ]
  • Xu X, Ke F. Designing a virtual-reality-based, gamelike math learning environment. American Journal of Distance Education. 2016; 30 (1):27–38. doi: 10.1080/08923647.2016.1119621. [ CrossRef ] [ Google Scholar ]
  • Yang JC, Chen CH, Jeng MC. Integrating video-capture virtual reality technology into a physically interactive learning environment for English learning. Computers & Education. 2010; 55 (3):1346–1356. doi: 10.1016/j.compedu.2010.06.005. [ CrossRef ] [ Google Scholar ]
  • Yerden, A., & Akkuş, N. (2020). Virtual reality remote access laboratory for teaching programmable logic controller topics.
  • Yilmaz RM, Topu FB, Takkaç Tulgar A. An examination of the studies on foreign language teaching in pre-school education: A bibliometric mapping analysis. Computer Assisted Language Learning. 2019 doi: 10.1080/09588221.2019.1681465. [ CrossRef ] [ Google Scholar ]
  • Zappatore, M., Longo, A., & Bochicchio, M. A. (2015). The bibliographic reference collection GRC2014 for the Online Laboratory Research community. In Proceedings of 2015 12th International Conference on Remote Engineering and Virtual Instrumentation (REV) (pp. 24–31). IEEE).
  • Zhang M, Zhang Z, Chang Y, Aziz ES, Esche S, Chassapis C. Recent developments in game-based virtual reality educational laboratories using the microsoft kinect. International Journal of Emerging Technologies in Learning (IJET) 2018; 13 (1):138–159. doi: 10.3991/ijet.v13i01.7773. [ CrossRef ] [ Google Scholar ]
  • Zhang X, Jiang S, Pablos P, Lytras MD, Sun Y. How virtual reality affects perceived learning effectiveness: A task– technology fit perspective. Behaviour y Information Technology. 2017; 36 (5):548–556. doi: 10.1080/0144929x.2016.1268647. [ CrossRef ] [ Google Scholar ]

Adaptive Shooting Disciplines: A Scoping Review of the Literature with Bibliometric Analysis

Affiliations.

  • 1 Department of Neuroscience, Rehabilitation, Ophthalmology, Genetics, Maternal and Child Health (DINOGMI), University of Genoa, 16132 Genoa, Italy.
  • 2 Orthopedics and Orthopedic Oncology, Department of Surgery, Oncology and Gastroenterology (DiSCOG), University of Padua, 35128 Padua, Italy.
  • 3 Physical Education and Sports Teaching Department, Kazim Karabekir Faculty of Education, Ataturk University, Erzurum 25030, Turkey.
  • 4 Laboratory for Industrial and Applied Mathematics (LIAM), Department of Mathematics and Statistics, York University, Toronto, ON M3J 1P3, Canada.
  • 5 Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS), Ospedale Policlinico San Martino, 16132 Genoa, Italy.
  • 6 Institut Supérieur du Sport et de l'Éducation Physique de Sfax, University of Sfax, Sfax 3000, Tunisia.
  • 7 Research Laboratory: Education, Motricity, Sport and Health, EM2S, LR19JS01, University of Sfax, Sfax 3000, Tunisia.
  • 8 Department of Clinical Research and Development, LUXMED Group, 02-676 Warsaw, Poland.
  • 9 Institute of Research Outcomes, Maria Sklodowska-Curie Medical Academy, 03-211 Warsaw, Poland.
  • PMID: 38391838
  • PMCID: PMC10888416
  • DOI: 10.3390/healthcare12040463

Para-archery and para-shooting, two very popular adaptive shooting disciplines that have earned their place as major official events in the Paralympic Games, share some similarities, as well as distinctive features in terms of rules, physiological requirements, and equipment used. The International Paralympic Committee has a clear responsibility to ensure that all sports within its jurisdiction, including adaptive shooting, can achieve excellence in their respective fields. To achieve this, the conduct of well-designed studies and rigorous research is essential. Although some research has been conducted in this area, a comprehensive and systematic evaluation is still needed. Therefore, the present study aims to provide a thorough review and synthesis of existing research on adaptive shooting disciplines, identify strengths and gaps, and suggest future directions. Arksey and O'Malley's methodology is leveraged and enhanced with bibliometric and policy analyses to review literature on adaptive shooting. Databases like PubMed/MEDLINE, Scopus, Web of Science, OvidSP, and EMBASE were searched, focusing on studies in adaptive shooting disciplines and analysing these findings through a blend of thematic and statistical methods. Twenty-four studies totalling 483 para-athletes (299 para-shooters and 184 para-archers) are included in this scoping review, focusing on a range of aspects, including physiological responses (n = 9), research design and measurement methods for evidence-based classification (n = 6), biopsychosocial aspects (n = 3), development of new methods and technologies (n = 4), kinematic and biomechanical assessment (n = 1), and epidemiology of injuries (n = 1). Seven articles focused exclusively on para-archery, thirteen exclusively on para-shooting, and four on both para-archery and para-shooting. In conclusion, the available literature on adaptive shooting disciplines is still very limited, especially regarding para-archery compared to para-shooting. This highlights the need for further research in many key areas to ensure a better understanding of the different disciplines and to provide appropriate support for para-athletes. Future research in para-archery and para-shooting should focus on technological innovations, biomechanical studies, and psychological support to enhance athlete performance and accessibility. Addressing the imbalance between the two disciplines, along with injury prevention and global participation, can drive the sports towards greater inclusivity and equity for para-athletes worldwide.

Keywords: biomechanical assessment; evidence-based classification; injuries; kinematic assessment; para-archery; para-shooting; physiological responses.

Publication types

Grants and funding.

Help | Advanced Search

Computer Science > Digital Libraries

Title: a literature review of literature reviews in pattern analysis and machine intelligence.

Abstract: By consolidating scattered knowledge, the literature review provides a comprehensive understanding of the investigated topic. However, excessive reviews, especially in the booming field of pattern analysis and machine intelligence (PAMI), raise concerns for both researchers and reviewers. In response to these concerns, this Analysis aims to provide a thorough review of reviews in the PAMI field from diverse perspectives. First, large language model-empowered bibliometric indicators are proposed to evaluate literature reviews automatically. To facilitate this, a meta-data database dubbed RiPAMI, and a topic dataset are constructed, which are utilized to obtain statistical characteristics of PAMI reviews. Unlike traditional bibliometric measurements, the proposed article-level indicators provide real-time and field-normalized quantified assessments of reviews without relying on user-defined keywords. Second, based on these indicators, the study presents comparative analyses of different reviews, unveiling the characteristics of publications across various fields, periods, and journals. The newly emerging AI-generated literature reviews are also appraised, and the observed differences suggest that most AI-generated reviews still lag behind human-authored reviews in several aspects. Third, we briefly provide a subjective evaluation of representative PAMI reviews and introduce a paper structure-based typology of literature reviews. This typology may improve the clarity and effectiveness for scholars in reading and writing reviews, while also serving as a guide for AI systems in generating well-organized reviews. Finally, this Analysis offers insights into the current challenges of literature reviews and envisions future directions for their development.

Submission history

Access paper:.

  • Download PDF
  • HTML (experimental)
  • Other Formats

license icon

References & Citations

  • Google Scholar
  • Semantic Scholar

BibTeX formatted citation

BibSonomy logo

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

  • Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

IMAGES

  1. (PDF) A Systematic Literature Review with Bibliometric Meta-Analysis of

    a literature review with bibliometric analysis

  2. (PDF) Geoheritage and Geosites: A Bibliometric Analysis and Literature

    a literature review with bibliometric analysis

  3. (PDF) A first bibliometric literature review on Metaverse

    a literature review with bibliometric analysis

  4. (PDF) Conducting systematic literature reviews and bibliometric analyses

    a literature review with bibliometric analysis

  5. Systematic Literature Review and Bibliometric Analysis(slr)(systematic

    a literature review with bibliometric analysis

  6. (PDF) Bibliometric analysis and review of Building Information

    a literature review with bibliometric analysis

VIDEO

  1. Bibliometric Analysis using Biblioshiny Software by Hiranya Dissanayake (Part II)

  2. Review of literature

  3. What is Literature Review?

  4. Part 2 Writing the Review of Literature

  5. Bibliometric analysis

  6. Workshop Systematic Literature Review (SLR) & Bibliometric Analysis

COMMENTS

  1. Conducting systematic literature reviews and bibliometric analyses

    1. Introduction Literature reviews play an essential role in academic research to gather existing knowledge and to examine the state of a field ( Cropanzano, 2009; Kunisch et al., 2018 ). Researchers typically collect available evidence on a topic or issue prior to conducting new research to assess the state of the already available evidence.

  2. How to conduct a bibliometric analysis: An overview and guidelines

    Bibliometric analysis is a popular and rigorous method for exploring and analyzing large volumes of scientific data. It enables us to unpack the evolutionary nuances of a specific field, while shedding light on the emerging areas in that field. Yet, its application in business research is relatively new, and in many instances, underdeveloped.

  3. A systematic literature review and bibliometric analysis based on

    A systematic literature review and bibliometric analysis based on member subscription services in retailing - ScienceDirect Journals & Books January-February 2024 A systematic literature review and bibliometric analysis based on member subscription services in retailing https://doi.org/10.1016/j.elerap.2023.101344

  4. Systematic literature review and bibliometric analysis on virtual

    The objective of this study is to identify and analyze the scientific literature with a bibliometric analysis to find the main topics, authors, sources, most cited articles, and countries in the literature on virtual reality in education.

  5. Guidelines for interpreting the results of bibliometric analysis: A

    Bibliometric analysis is an analytical technique that is often employed in systematic literature reviews—it involves the quantitative analysis of scholarly works (Donthu et al., 2021; Kraus et al., 2022; Lim, Kumar, et al., 2022; Mukherjee et al., 2022; Paul, Lim, et al., 2021 ).

  6. Financial literacy: A systematic review and bibliometric analysis

    Given the paucity of comprehensive summaries in the extant literature, this systematic review, coupled with bibliometric analysis, endeavours to take a meticulous approach intended at presenting quantitative and qualitative knowledge on the ever-emerging subject of financial literacy.

  7. Full article: Evaluating literature with bibliometrics

    Bibliometric analysis has really taken off in the last few years. From about 1994 onwards, the information science domain has been teeming with the developments this kind of computer-assisted review methodology can offer and now increasingly other subject disciplines have adopted this approach.

  8. (PDF) How undertake a literature review through Bibliometrics. An

    Bibliometric analysis is a methodology generally applied to draw a vast picture (Porter et al., 2002) in a literature review. ... Therefore, the study applied a literature review in a bibliometric ...

  9. Work-life balance: a systematic literature review and bibliometric analysis

    Purpose. The purpose of this paper is to provide a clear view of current dynamics and research diversification of extant literature in the field of work-life balance (WLB). This paper provides a systematic and critical analysis of WLB literature using bibliometric analysis.

  10. Work-life balance: a systematic literature review and bibliometric analysis

    This paper provides a systematic and critical analysis of WLB literature using bibliometric analysis. Design/methodology/approach Scopus database has been used for carrying out this...

  11. Preliminary guideline for reporting bibliometric reviews of the

    A bibliometric review of the biomedical literature could be essential in synthesizing evidence if thoroughly conducted and documented. Although very similar to review papers in nature, it slightly differs in synthesizing the data when it comes to providing a pile of evidence from different studies into a single document.

  12. Literature review and bibliometric analysis on data-driven assessment

    This analysis is based on the Web of Science, Clarivate Analytics database and provides insights into the transient characteristics and research trends within published spatial landslide assessments. Following the bibliometric analysis, a more detailed review of the most recent publications from 1985 to 2020 is given.

  13. Conducting systematic literature reviews and bibliometric analyses

    In general terms, the bibliometric analysis represents an essential reference source for developing new research or even deepening scientific fields (Santana et al., 2021;Vasconcelos et al., 2020 ...

  14. Sustainable development goals: a bibliometric analysis of literature

    Sustainable development goals: a bibliometric analysis of literature reviews Review Article Published: 24 November 2022 Volume 30 , pages 5502-5515, ( 2023 ) Cite this article Download PDF Aims and scope Submit manuscript Natália Ueda Yamaguchi, Eduarda Gameleira Bernardino, Maria Eliana Camargo Ferreira, Bruna Pietroski de Lima,

  15. Digital labour: a systematic literature review and bibliometric analysis

    This study aims to conduct a systematic literature review and bibliometric analysis on the topic of digital labour. The study also identifies the future research directions for the topic.,In total, 118 research papers were identified and reviewed from 11 established research databases and A*, A and B category journals from the ABDC journal list.

  16. Bibliometrics and systematic reviews: A comparison between the Proknow

    This research aims to analyze and compare the selection of relevant papers by two bibliometric methods, namely, the Knowledge Development Process-Constructivist (Proknow-C) and the Methodi Ordinatio, which were also discussed in terms of common steps, limitations and underlying assumptions.

  17. The Ebb and Flow of Brand Loyalty: A 28-Year Bibliometric and Content

    Business research is facing the challenge of scattered knowledge, particularly in the realm of brand loyalty (BL). Although literature reviews on BL exist, they predominantly concentrate on the pre-sent state, neglecting future trends. Therefore, a comprehensive review is imperative to ascertain emerging trends in BL This study employs a bibliometric approach, analyzing 1,468 papers from the ...

  18. Systematic literature review and bibliometric analysis on virtual

    This article is organized as follows: the first part introduces the study, including the objectives and the research questions. A second section presents a review of the literature on bibliometric analysis. The methodology used is defined in the third section, indicating the search procedures used to identify the literature on VR in education.

  19. Corporate Sustainability Practices: A Systematic Literature Review and

    A bibliometric analysis cum systematic literature review was conducted on 116 articles selected from the Scopus database. Corporate sustainability research has gained popularity since 2015, and most studies were undertaken in developed countries.

  20. Uncover the trends, gaps, and main topics on online grocery ...

    The bibliometric analysis in this study aims to find trending topics and research gaps in online grocery shopping. At the eligibility stage, there are 170 full-text articles related to the issue of online grocery shopping; until the last stage, through manual selection, the researcher determined 43 full-text articles that met the qualifications ...

  21. A Synthetic Review on Enterprise Digital Transformation: A Bibliometric

    The digital transformation of enterprises has become an inevitable trend. Conducting a bibliometric analysis of the literature on the digital transformation of enterprises and exploring the current status of digital transformation will provide valuable references and suggestions for enterprises. Both bibliometric analysis and literature review methods are utilized in this study.

  22. The Ebb and Flow of Brand Loyalty: A 28-Year Bibliometric ...

    Business research is facing the challenge of scattered knowledge, particularly in the realm of brand loyalty (BL). Although literature reviews on BL exist, they predominantly concentrate on the pre-sent state, neglecting future trends. Therefore, a comprehensive review is imperative to ascertain emerging trends in BL This study employs a bibliometric approach, analyzing 1,468 papers from the ...

  23. Global trends of waste-to-energy (WtE) technologies in ...

    Bibliometric methods are widely used because of their advantages in qualitative and quantitative literature information analysis. A comprehensive search was conducted in the Web of Science (WOS) Core Collection database, covering the period from 1990-2022, resulting in the collection of 702 articles.

  24. Adaptive Shooting Disciplines: A Scoping Review of the Literature with

    Therefore, the present study aims to provide a thorough review and synthesis of existing research on adaptive shooting disciplines, identify strengths and gaps, and suggest future directions. Arksey and O'Malley's methodology is leveraged and enhanced with bibliometric and policy analyses to review literature on adaptive shooting.

  25. Sustainability Accounting: A Systematic Literature Review and

    Information retrieval for bibliometric analysis was conducted in two scientometric databases Scopus Elsevier and Web of Science Core Collection for the keyword "sustainability accounting" in the ...

  26. How to conduct a bibliometric analysis: An overview and guidelines

    Bibliometric analysis is a popular and rigorous method for exploring and analyzing large volumes of scientific data. It enables us to unpack the evolutionary nuances of a specific field, while shedding light on the emerging areas in that field. Yet, its application in business research is relatively new, and in many instances, underdeveloped.

  27. Global acute-on-chronic liver failure trends during 2012-2022: A

    Introduction. Acute-on-chronic liver failure (ACLF) is a clinical syndrome with high short-term mortality. ACLF has been increasingly studied in recent years; however, a bibliometric analysis of the entire ACLF field has not been conducted. This study assesses current global trends and hotspots in ACLF research. Materials and methods. The core Web of Science database was searched for all ACLF ...

  28. A Literature Review of Literature Reviews in Pattern Analysis and

    By consolidating scattered knowledge, the literature review provides a comprehensive understanding of the investigated topic. However, excessive reviews, especially in the booming field of pattern analysis and machine intelligence (PAMI), raise concerns for both researchers and reviewers. In response to these concerns, this Analysis aims to provide a thorough review of reviews in the PAMI ...