Advertisement

Advertisement

Early Childhood Educators’ Well-Being: An Updated Review of the Literature

  • Published: 24 August 2016
  • Volume 45 , pages 583–593, ( 2017 )

Cite this article

  • Tamara Cumming 1  

14k Accesses

121 Citations

2 Altmetric

Explore all metrics

Researchers are increasingly recognising the connections between early childhood educators’ well-being and their capacity for providing high quality education and care. The past five years have seen an intensification of research concerning early childhood educators’ well-being. However, fragmentation along conceptual, contextual and methodological lines makes it difficult to clearly identify the most effective focus for future research. The purpose of this article is to identify trends in, and implications of recent research concerned with educators’ well-being. Attention is given to ways recent studies address concerns raised in a review of earlier literature (Hall-Kenyon et al. in Early Child Educ J 42(3):153–162, 2014 , doi: 10.1007/s10643-013-0595-4 ), and what implications recent studies have for future research efforts concerned with educators’ well-being.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

As explained in the search procedures, crossover with the era of literature reviewed by Hall-Kenyon et al. ( 2014 ) was done deliberately. There is no duplication of studies reviewed between this current review and that of Hall-Kenyon et al.

It is possible that, despite searching numerous databases, there may be more studies concerned with educators’ well-being than have been covered in this review and in Hall-Kenyon et al.’s ( 2014 ). A search using the term ‘early childhood educators’ in journal articles published prior to 2012, or the inclusion of book chapters or dissertations could potentially expand the body of literature further.

Two articles by Rentzou reported on data from the same study, therefore I have counted one study only for this author.

Abu Taleb, T. F. (2013). Job satisfaction among Jordan’s kindergarten teachers: Effects of workplace conditions and demographic characteristics. Early Childhood Education Journal, 41 , 143–152. doi: 10.1007/s10643-012-0526-9 .

Article   Google Scholar  

Andrew, Y. (2015). What we feel and what we do: Emotional capital in early childhood work. Early Years: An International Research Journal, 35 (4), 351–365. doi: 10.1080/09575146.2015.1077206 .

Australian Children’s Services Education & Care Quality Authority. (2012). The National Quality Framework. http://acecqa.gov.au/national-quality-framework/introducing-the-national-quality-framework .

Boyd, M. (2013). I love my work but…”The professionalization of early childhood education. The Qualitative Report, 18 , 1–20. http://www.nova.edu.ssss/QR/QR18/boyd71.pdf .

Bullough, R. V., Jr. (2015). Teaming and teaching in ECE: Neoliberal reforms, teacher metaphors, and identity in Head Start. Journal of Research in Childhood Education, 29 , 410–417. doi: 10.1080/02568543.2015.1050563 .

Cassidy, D. J., Lower, J. K., Kintner-Duffy, V. L., Hegde, A. V., & Shim, J. (2011). The day-to-day reality of teacher turnover in preschool classrooms: An analysis of classroom context and teacher, director, and parent perspectives. Journal of Research in Childhood Education , 25 (1), 1–23. doi: 10.1080/02568543.2011.533118 .

Center for for the Study of Child Care Employment. (2014). Supportive environment quality underlying adult learning. www.irle.berkeley.edu/cscce/?s=Sequal .

Cheng, J.-N., & Chen, Y. (2011). The empirical study of the kindergarten teachers’ job satisfaction in Taiwan: Exploring the effect of the intrinsic demand, external reward, and organizational treatment. The Journal of Human Resource and Adult Learning, 7 (2), 127–132.

Google Scholar  

Cheruvu, R., Souto-Manning, M., Lencl, T., & Chin-Calubaquib, M. (2015). Race, isolation, and exclusion: What early childhood teacher educators need to know about the experiences of pre-service teachers of color. Urban Review , 47 , 237–265. doi: 10.1077/s11256-014-0291-8 .

Cheuk, W. H., Wong, K. S., & Rosen, S. (2011). The effects of being spurned and self-esteem on depersonalization and coping preferences in kindergarten teachers: The case of Hong Kong. Social Psychology of Education, 14 (1), 57–73. doi: 10.1007/s11218-010-9138-7 .

Corr, L., Cook, K., LaMontagne, A. D., Waters, E., & Davis, E. (2015). Associations between Australian early childhood educators’ mental health and working conditions. Australasian Journal of Early Childhood, 40 (3), 69–78.

Corr, L., Davis, E., Cook, K., Waters, E., & LaMontagne, A. D. (2014). Fair relationships and policies to support family day care educators’ mental health: A qualitative study. BMC Public Health . doi: 10.1186/1471-2458-14-1214 .

Cumming, T. (2015). Early childhood educators’ experiences in their work environments: Shaping (im)possible ways of being an educator? Complicity: An International Journal of Complexity and Education, 12 (1), 52–66.

Curbow, B., Spratt, K., Ungaretti, A., McDonnell, K., & Breckler, S. (2000). Development of the child care worker job stress inventory. Early Childhood Research Quarterly, 15 , 515–536.

de Schipper, E. J., Riksen-Walraven, J. M., Geurts, S. A. E., & Derksen, J. J. L. (2008). General mood of professional caregivers in child care centers and the quality of caregiver-child interactions. Journal of Research in Personality, 42 , 515–526. doi: 10.1016/j.jrp.2007.07.009 .

Erdiller, Z. B., & Doğan, Ö. (2015). The examination of teacher stress among Turkish early childhood education teachers. Early Child Development and Care , 185 (4), 631–646. doi: 10.1080/03004430.2014.946502 .

Faulkner, M., Gerstenblatt, P., Lee, A., Vallejo, V., & Travis, D. (2014). Childcare providers: Work stress and personal well-being. Journal of Early Childhood Research . doi: 10.1177/1476718X14552871 .

Foucault, M. (1980). Power/knowledge . Brighton: Harvester.

Hall-Kenyon, K. M., Bullough, R. V., MacKay, K. L., & Marshall, E. E. (2014). Preschool teacher well-being: A review of the literature. Early Childhood Education Journal, 42 (3), 153–162. doi: 10.1007/s10643-013-0595-4 .

Hur, E., Jeon, L., & Buettner, C. K. (2016). Preschool teachers’ child-centred beliefs: Direct and indirect associations with work climate and job-related wellbeing. Child Youth Care Forum , 45 (3), 451–465. doi: 10.1007/s10566-015-9338-6 .

International Labour Organization. (2014). ILO Policy Guidelines on the promotion of decent work for early childhood education personnel . Geneva.

Jennings, P. A. (2015). Early childhood teachers’ well-being, mindfulness and self-compassion in relation to classroom quality and attitudes towards challenging students. Mindfulness, 6 , 732–743. doi: 10.1007/s12671-014-0312-4 .

Jeon, L., Buettner, C. K., & Hur, E. (2016). Preschool teachers’ professional background, process quality, and job attitudes: A person-centred approach. Early Education & Development , 27 (4), 551–571. doi: 10.1080/10409289.2016.1099354 .

Jorde-Bloom, P. (1988). Closing the gap: An analysis of teacher and administrator perceptions of organizational climate in the early childhood setting. Teaching and Teacher Education , 4 (2), 111–120.

Jovanovic, J. (2013). Retaining early childhood educators. Gender, Work and Organization, 20 (5), 528–544. doi: 10.1111/j.1468-0432.2012.00602.x .

Kilderry, A. (2015). The intensification of performativity in early childhood education. Journal of Curriculum Studies, 47 (5), 633–652. doi: 10.1080/00220272.2015.1052850 .

King, E. K., Johnson, A. V., Cassidy, D. J., Wang, Y. C., Lower, J. K., & Kintner-Duffy, V. L. (2015). Preschool teachers’ financial well-being and work time supports: Associations with children’s emotional expressions and behaviors in classrooms. Early Childhood Education Journal . doi: 10.1007/s10643-015-0744-z .

Kotaman, H. (2014). Turkish early childhood teachers’ emotional problems in early years of their professional lives. European Early Childhood Education Research Journal . doi: 10.1080/1350293X.2014.970849 .

Kusma, B., Groneberg, D. A., Nienhaus, A., & Mache, S. (2012). Determinants of day care teachers’ job satisfaction. Central European Journal of Public Health , 20 (3), 191–198.

Løvgren, M. (2016). Emotional exhaustion in day-care workers. European Early Childhood Education Research Journal, 24 (1), 157–167. doi: 10.1080/1350293X.2015.1120525 .

Nislin, M. A., Sajaniemi, N. K., Sims, M., Suhonen, E., Maldonado Montero, E. F., Hirvonen, A., et al. (2016). Pedagogical work, stress regulation and work-related well-being among early childhood professionals in integrated special day-care groups. European Journal of Special Needs Education, 31 (1), 27–43. doi: 10.1080/08856257.2015.1087127 .

Ota, C. L., Baumgartner, J. L., & Berghout Austin, A. M. (2013). Provider stress and children’s active engagement. Journal of Research in Childhood Education , 27 , 61–73. doi: 10.1080/02568543.2012.739588 .

Rentzou, K. (2012). Examination of work environment factors relating to burnout syndrome of early childhood educators in Greece. Child Care in Practice, 18 (2), 165–181. doi: 10.1080/13575279.2012.657609 .

Royer, N., & Moreau, C. (2015). A survey of Canadian early childhood educators’ psychological wellbeing at work. Early Childhood Education Journal , 44 (2), 135–146. doi: 10.1007/s10643-015-0696-3 .

Schreyer, I., & Krause, M. (2016). Pedagogical staff in children’s day care centres in Germany—Links between working conditions, job satisfaction, commitment and work-related stress. Early Years An International Research Journal . doi: 10.1080/09575146.2015.1115390 .

Wagner, S. L., Forer, B., Cepeda, I. L., Goelman, H., Maggi, S., D’Angiulli, A., et al. (2013). Perceived stress and Canadian early childcare educators. Child and Youth Care Forum, 42 , 53–70. doi: 10.1007/s10566-012-9187-5 .

Whitaker, R., Dearth-Wesley, T., & Gooze, R. A. (2015). Workplace stress and the quality of teacher-children relationships in Head Start. Early Childhood Research Quarterly, 30 , 57–69. doi: 10.1016/j.ecresq.2014.08.008 .

Whitebook, M., Howes, C., & Phillips, D. (1990). The national child care staffing study. Final report: Who Cares? Child care teachers and the quality of care in America . Washington, DC: Center for the Child Care Workforce.

Whitebook, M., & Ryan, S. (2011). Degrees in context: Asking the right questions about preparing skilled and effective teachers of young children . Rutgers, NJ: National Institute for Early Education Research.

Download references

Acknowledgments

The author wishes to thank Professor Jennifer Sumsion for her generous advice and feedback on drafts of this article, as well as the helpful suggestions of the anonymous reviewers.

Author information

Authors and affiliations.

Charles Sturt University, Suite 1.01, Quad 3, Bennelong Parkway, Sydney Olympic Park, Sydney, NSW, 2127, Australia

Tamara Cumming

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Tamara Cumming .

Rights and permissions

Reprints and permissions

About this article

Cumming, T. Early Childhood Educators’ Well-Being: An Updated Review of the Literature. Early Childhood Educ J 45 , 583–593 (2017). https://doi.org/10.1007/s10643-016-0818-6

Download citation

Published : 24 August 2016

Issue Date : September 2017

DOI : https://doi.org/10.1007/s10643-016-0818-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Early childhood educators
  • Early childhood teachers
  • Childcare workers
  • Find a journal
  • Publish with us
  • Track your research
  • - Google Chrome

Intended for healthcare professionals

  • Access provided by Google Indexer
  • My email alerts
  • BMA member login
  • Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Home

Search form

  • Advanced search
  • Search responses
  • Search blogs
  • When and how to update...

When and how to update systematic reviews: consensus and checklist

  • Related content

Peer review

This article has a correction. please see:.

  • Errata - September 06, 2016
  • Paul Garner , professor 1 ,
  • Sally Hopewell , associate professor 2 ,
  • Jackie Chandler , methods coordinator 3 ,
  • Harriet MacLehose , senior editor 3 ,
  • Elie A Akl , professor 5 6 ,
  • Joseph Beyene , associate professor 7 ,
  • Stephanie Chang , director 8 ,
  • Rachel Churchill , professor 9 ,
  • Karin Dearness , managing editor 10 ,
  • Gordon Guyatt , professor 4 ,
  • Carol Lefebvre , information consultant 11 ,
  • Beth Liles , methodologist 12 ,
  • Rachel Marshall , editor 3 ,
  • Laura Martínez García , researcher 13 ,
  • Chris Mavergames , head 14 ,
  • Mona Nasser , clinical lecturer in evidence based dentistry 15 ,
  • Amir Qaseem , vice president and chair 16 17 ,
  • Margaret Sampson , librarian 18 ,
  • Karla Soares-Weiser , deputy editor in chief 3 ,
  • Yemisi Takwoingi , senior research fellow in medical statistics 19 ,
  • Lehana Thabane , director and professor 4 20 ,
  • Marialena Trivella , statistician 21 ,
  • Peter Tugwell , professor of medicine, epidemiology, and community medicine 22 ,
  • Emma Welsh , managing editor 23 ,
  • Ed C Wilson , senior research associate in health economics 24 ,
  • Holger J Schünemann , professor 4 5
  • 1 Cochrane Infectious Diseases Group, Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool L3 5QA, UK
  • 2 Oxford Clinical Trials Research Unit, University of Oxford, Oxford, UK
  • 3 Cochrane Editorial Unit, Cochrane Central Executive, London, UK
  • 4 Department of Clinical Epidemiology and Biostatistics and Department of Medicine, McMaster University, Hamilton, ON, Canada
  • 5 Cochrane GRADEing Methods Group, Ottawa, ON, Canada
  • 6 Department of Internal Medicine, American University of Beirut, Beirut, Lebanon
  • 7 Department of Mathematics and Statistics, McMaster University
  • 8 Evidence-based Practice Center Program, Agency for Healthcare and Research Quality, Rockville, MD, USA
  • 9 Centre for Reviews and Dissemination, University of York, York, UK
  • 10 Cochrane Upper Gastrointestinal and Pancreatic Diseases Group, Hamilton, ON, Canada
  • 11 Lefebvre Associates, Oxford, UK
  • 12 Kaiser Permanente National Guideline Program, Portland, OR, USA
  • 13 Iberoamerican Cochrane Centre, Barcelona, Spain
  • 14 Cochrane Informatics and Knowledge Management, Cochrane Central Executive, Freiburg, Germany
  • 15 Plymouth University Peninsula School of Dentistry, Plymouth, UK
  • 16 Department of Clinical Policy, American College of Physicians, Philadelphia, PA, USA
  • 17 Guidelines International Network, Pitlochry, UK
  • 18 Children’s Hospital of Eastern Ontario, Ottawa, ON, Canada
  • 19 Institute of Applied Health Research, University of Birmingham, Birmingham, UK
  • 20 Biostatistics Unit, Centre for Evaluation, McMaster University, Hamilton, ON, Canada
  • 21 Centre for Statistics in Medicine, University of Oxford, Oxford, UK
  • 22 University of Ottawa, Ottawa, ON, Canada
  • 23 Cochrane Airways Group, Population Health Research Institute, St George’s, University of London, London, UK
  • 24 Cambridge Centre for Health Services Research, University of Cambridge, Cambridge, UK
  • Correspondence to: P Garner Paul.Garner{at}lstmed.ac.uk
  • Accepted 26 May 2016

Updating of systematic reviews is generally more efficient than starting all over again when new evidence emerges, but to date there has been no clear guidance on how to do this. This guidance helps authors of systematic reviews, commissioners, and editors decide when to update a systematic review, and then how to go about updating the review.

Systematic reviews synthesise relevant research around a particular question. Preparing a systematic review is time and resource consuming, and provides a snapshot of knowledge at the time of incorporation of data from studies identified during the latest search. Newly identified studies can change the conclusion of a review. If they have not been included, this threatens the validity of the review, and, at worst, means the review could mislead. For patients and other healthcare consumers, this means that care and policy development might not be fully informed by the latest research; furthermore, researchers could be misled and carry out research in areas where no further research is actually needed. 1 Thus, there are clear benefits to updating reviews, rather than duplicating the entire process as new evidence emerges or new methods develop. Indeed, there is probably added value to updating a review, because this will include taking into account comments and criticisms, and adoption of new methods in an iterative process. 2 3 4 5 6

Cochrane has over 20 years of experience with preparing and updating systematic reviews, with the publication of over 6000 systematic reviews. However, Cochrane’s principle of keeping all reviews up to date has not been possible, and the organisation has had to adapt: from updating when new evidence becomes available, 7 to updating every two years, 8 to updating based on need and priority. 9 This experience has shown that it is not possible, sensible, or feasible to continually update all reviews all the time. Other groups, including guideline developers and journal editors, adopt updating principles (as applied, for example, by the Systematic Reviews journal; https://systematicreviewsjournal.biomedcentral.com/ ).

The panel for updating guidance for systematic reviews (PUGs) group met to draw together experiences and identify a common approach. The PUGs guidance can help individuals or academic teams working outside of a commissioning agency or Cochrane, who are considering writing a systematic review for a journal or to prepare for a research project. The guidance could also help these groups decide whether their effort is worthwhile.

Summary points

Updating systematic reviews is, in general, more efficient than starting afresh when new evidence emerges. The panel for updating guidance for systematic reviews (PUGs; comprising review authors, editors, statisticians, information specialists, related methodologists, and guideline developers) met to develop guidance for people considering updating systematic reviews. The panel proposed the following:

Decisions about whether and when to update a systematic review are judgments made for individual reviews at a particular time. These decisions can be made by agencies responsible for systematic review portfolios, journal editors with systematic review update services, or author teams considering embarking on an update of a review.

The decision needs to take into account whether the review addresses a current question, uses valid methods, and is well conducted; and whether there are new relevant methods, new studies, or new information on existing included studies. Given this information, the agency, editors, or authors need to judge whether the update will influence the review findings or credibility sufficiently to justify the effort in updating it.

Review authors and commissioners can use a decision framework and checklist to navigate and report these decisions with “update status” and rationale for this status. The panel noted that the incorporation of new synthesis methods (such as Grading of Recommendations Assessment, Development and Evaluation (GRADE)) is also often likely to improve the quality of the analysis and the clarity of the findings.

Given a decision to update, the process needs to start with an appraisal and revision of the background, question, inclusion criteria, and methods of the existing review.

Search strategies should be refined, taking into account changes in the question or inclusion criteria. An analysis of yield from the previous edition, in relation to databases searched, terms, and languages can make searches more specific and efficient.

In many instances, an update represents a new edition of the review, and authorship of the new version needs to follow criteria of the International Committee of Medical Journal Editors (ICMJE). New approaches to publishing licences could help new authors build on and re-use the previous edition while giving appropriate credit to the previous authors.

The panel also reflected on this guidance in the context of emerging technological advances in software, information retrieval, and electronic linkage and mining. With good synthesis and technology partnerships, these advances could revolutionise the efficiency of updating in the coming years.

Panel selection and procedures

An international panel of authors, editors, clinicians, statisticians, information specialists, other methodologists, and guideline developers was invited to a two day workshop at McMaster University, Hamilton, Canada, on 26-27 June 2014, organised by Cochrane. The organising committee selected the panel (web appendix 1). The organising committee invited participants, put forward the agenda, collected background materials and literature, and drafted the structure of the report.

The purpose of the workshop was to develop a common approach to updating systematic reviews, drawing on existing strategies, research, and experience of people working in this area. The selection of participants aimed on broad representation of different groups involved in producing systematic reviews (including authors, editors, statisticians, information specialists, and other methodologists), and those using the reviews (guideline developers and clinicians). Participants within these groups were selected on their expertise and experience in updating, in previous work developing methods to assess reviews, and because some were recognised for developing approaches within organisations to manage updating strategically. We sought to identify general approaches in this area, and not be specific to Cochrane; although inevitably most of the panel were somehow engaged in Cochrane.

The workshop structure followed a series of short presentations addressing key questions on whether, when, and how to update systematic reviews. The proceedings included the management of authorship and editorial decisions, and innovative and technological approaches. A series of small group discussions followed each question, deliberating content, and forming recommendations, as well as recognising uncertainties. Large group, round table discussions deliberated further these small group developments. Recommendations were presented to an invited forum of individuals with varying levels of expertise in systematic reviews from McMaster University (of over 40 people), widely known for its contributions to the field of research evidence synthesis. Their comments helped inform the emerging guidance.

The organising committee became the writing committee after the meeting. They developed the guidance arising from the meeting, developed the checklist and diagrams, added examples, and finalised the manuscript. The guidance was circulated to the larger group three times, with the PUGs panel providing extensive feedback. This feedback was all considered and carefully addressed by the writing committee. The writing committee provided the panel with the option of expressing any additional comments from the general or specific guidance in the report, and the option for registering their own view that might differ to the guidance formed and their view would be recorded in an annex. In the event, consensus was reached, and the annex was not required.

Definition of update

The PUGs panel defined an update of a systematic review as a new edition of a published systematic review with changes that can include new data, new methods, or new analyses to the previous edition. This expands on a previous definition of a systematic review update. 10 An update asks a similar question with regard to the participants, intervention, comparisons, and outcomes (PICO) and has similar objectives; thus it has similar inclusion criteria. These inclusion criteria can be modified in the light of developments within the topic area with new interventions, new standards, and new approaches. Updates will include a new search for potentially relevant studies and incorporate any eligible studies or data; and adjust the findings and conclusions as appropriate. Box 1 provides some examples.

Box 1: Examples of what factors might change in an updated systematic review

A systematic review of steroid treatment in tuberculosis meningitis used GRADE methods and split the composite outcome in the original review of death plus disability into its two components. This improved the clarity of the reviews findings in relation to the effects and the importance of the effects of steroids on death and on disability. 11

A systematic review of dihydroartemisinin-piperaquine (DHAP) for treating malaria was updated with much more detailed analysis of the adverse effect data from the existing trials as a result of questions raised by the European Medicines Agency. Because the original review included other comparisons, the update required extracting only the DHAP comparisons from the original review, and a modification of the title and the PICO. 12

A systematic review of atorvastatin was updated with simple uncontrolled studies. 13 This update allowed comparisons with trials and strengthened the review findings. 14

Which systematic reviews should be updated and when?

Any group maintaining a portfolio of systematic reviews as part of their normative work, such as guidelines panels or Cochrane review groups, will need to prioritise which reviews to update. Box 2 presents the approaches used by the Agency for HealthCare Research and Quality (AHRQ) and Cochrane to prioritise which systematic reviews to update and when. Clearly, the responsibility for deciding which systematic reviews should be updated and when they will be updated will vary: it may be centrally organised and resourced, as with the AHRQ scientific resource centre (box 2). In Cochrane, the decision making process is decentralised to the Cochrane Review Group editorial team, with different approaches applied, often informally.

Box 2: Examples of how different organisations decide on updating systematic reviews

Agency for healthcare research and quality (us).

The AHRQ uses a needs based approach; updating systematic reviews depends on an assessment of several criteria:

Stakeholder impact

Interest from stakeholder partners (such as consumers, funders, guideline developers, clinical societies, James Lind Alliance)

Use and uptake (for example, frequency of citations and downloads)

Citation in scientific literature including clinical practice guidelines

Currency and need for update

New research is available

Review conclusions are probably dated

Update decision

Based on the above criteria, the decision is made to either update, archive, or continue surveillance.

Of over 50 Cochrane editorial teams, most but not all have some systems for updating, although this process can be informal and loosely applied. Most editorial teams draw on some or all of the following criteria:

Strategic importance

Is the topic a priority area (for example, in current debates or considered by guidelines groups)?

Is there important new information available?

Practicalities in organising the update that many groups take into account

Size of the task (size and quality of the review, and how many new studies or analyses are needed)

Availability and willingness of the author team

Impact of update

New research impact on findings and credibility

Consider whether new methods will improve review quality

Priority to update, postpone update, class review as no longer requiring an update

The PUGs panel recommended an individualised approach to updating, which used the procedures summarised in figure 1 ⇓ . The figure provides a status category, and some options for classifying reviews into each of these categories, and builds on a previous decision tool and earlier work developing an updating classification system. 15 16 We provide a narrative for each step.

Fig 1 Decision framework to assess systematic reviews for updating, with standard terms to report such decisions

  • Download figure
  • Open in new tab
  • Download powerpoint

Step 1: assess currency

Does the published review still address a current question.

An update is only worthwhile if the question is topical for decision making for practice, policy, or research priorities (fig 1 ⇑ ). For agencies, people responsible for managing a portfolio of systematic reviews, there is a need to use both formal and informal horizon scanning. This type of scanning helps identify questions with currency, and can help identify those reviews that should be updated. The process could include monitoring policy debates around the review, media outlets, scientific (and professional) publications, and linking with guideline developers.

Has the review had good access or use?

Metrics for citations, article access and downloads, and sharing via social or traditional media can be used as proxy or indicators for currency and relevance of the review. Reviews that are widely cited and used could be important to update should the need arise. Comparable reviews that are never cited or rarely downloaded, for example, could indicate that they are not addressing a question that is valued, and might not be worth updating.

In most cases, updated reviews are most useful to stakeholders when there is new information or methods that result in a change in findings. However, there are some circumstances in which an up to date search for information is important for retaining the credibility of the review, regardless of whether the main findings would change or not. For example, key stakeholders would dismiss a review if a study is carried out in a relevant geographical setting but is not included; if a large, high profile study that might not change the findings is not included; or if an up to date search is required for a guideline to achieve credibility. Box 3 provides such examples. If the review does not answer a current question, the intervention has been superseded, then a decision can be made not to update and no further intelligence gathering is required (fig 1 ⇑ ).

Box 3: Examples of a systematic review’s currency

The public is interested in vitamin C for preventing the common cold: the Cochrane review includes over 29 trials with either no or small effects, concluding good evidence of no important effects. 17 Assessment: still a current question for the public.

Low osmolarity oral rehydration salt (ORS) solution versus standard solution for acute diarrhoea in children: the 2001 Cochrane review 18 led the World Health Organization to recommend ORS solution formula worldwide to follow the new ORS solution formula 19 and this has now been accepted globally. Assessment: no longer a current question.

Routine prophylactic antibiotics with caesarean section: the Cochrane review reports clear evidence of maternal benefit from placebo controlled trials but no information on the effects on the baby. 20 Assessment: this is a current question.

A systematic review published in the Lancet examined the effects of artemisinin based combination treatments compared with monotherapy for treating malaria and showed clear benefit. 21 Assessment: this established the treatment globally and is no longer a current question and no update is required.

A Cochrane review of amalgam restorations for dental caries 22 is unlikely to be updated because the use of dental amalgam is declining, and the question is not seen as being important by many dental specialists. Assessment: no longer a current question.

Did the review use valid methods and was it well conducted?

If the question is current and clearly defined, the systematic review needs to have used valid methods and be well conducted. If the review has vague inclusion criteria, poorly articulated outcomes, or inappropriate methods, then updating should not proceed. If the question is current, and the review has been cited or used, then it might be appropriate to simply start with a new protocol. The appraisal should take into account the methods in use when the review was done.

Step 2: identify relevant new methods, studies, and other information

Are there any new relevant methods.

If the question is current, but the review was done some years ago, the quality of the review might not meet current day standards. Methods have advanced quickly, and data extraction and understanding of the review process have become more sophisticated. For example:

Methods for assessing risk of bias of randomised trials, 23 diagnostic test accuracy (QUADAS-2), 24 and observational studies (ROBINS-1). 25

Application of summary of findings, evidence profiles, and related GRADE methods has meant the characteristics of the intervention, characteristics of the participants, and risk of bias are more thoroughly and systematically documented. 26 27

Integration of other study designs containing evidence, such economic evaluation and qualitative research. 28

There are other incremental improvements in a wide range of statistical and methodological areas, for example, in describing and taking into account cluster randomised trials. 29 AMSTAR can assess the overall quality of a systematic review, 30 and the ROBIS tool can provide a more detailed assessment of the potential for bias. 31

Are there any new studies or other information?

If an authoring or commissioning team wants to ensure that a particular review is up to date, there is a need for routine surveillance for new studies that are potentially relevant to the review, by searching and trial register inspection at regular intervals. This process has several approaches, including:

Formal surveillance searching 32

Updating the full search strategies in the original review and running the searches

Tracking studies in clinical trial and other registers

Using literature appraisal services 33

Using a defined abbreviated search strategy for the update 34

Checking studies included in related systematic reviews. 35

How often this surveillance is done, and which approaches to use, depend on the circumstances and the topic. Some topics move quickly, and the definition of “regular intervals” will vary according to the field and according to the state of evidence in the field. For example, early in the life of a new intervention, there might be a plethora of studies, and surveillance would be needed more frequently.

Step 3: assess the effect of updating the review

Will the adoption of new methods change the findings or credibility.

Editors, referees, or experts in the topic area or methodologists can provide an informed view of whether a review can be substantially improved by application of current methodological expectations and new methods (fig 1 ⇑ ). For example, a Cochrane review of iron supplementation in malaria concluded that there was “no significant difference between iron and placebo detected.” 36 An update of the review included a GRADE assessment of the certainty of the evidence, and was able to conclude with a high degree of certainty that iron does not cause an excess of clinical malaria because the upper relative risk confidence intervals of harm was 1.0 with high certainty of evidence. 37

Will the new studies, information, or data change the findings or credibility?

The assessment of new data contained in new studies and how these data might change the review is often used to determine whether an update should go ahead, and the speed with which the update should be conducted. The appraisal of these new data can be carried out in different ways. Initially, methods focused on statistical approaches to predict an overturning of the current review findings in terms of the primary or desired outcome (table 1 ⇓ ). Although this aspect is important, additional studies can add important information to a review, which is more than just changing the primary outcome to a more accurate and reliable estimate. Box 4 gives examples.

Formal prediction tools: how potentially relevant new studies can affect review conclusions

  • View inline

Box 4: Examples of new information other than new trials being important

The iconic Cochrane review of steroids in preterm labour was thought to provide evidence of benefit in infants, and this question no longer required new trials. However, a new large trial published in the Lancet in 2015 showed that in low and middle income countries, strategies to promote the uptake of neonatal steroids increased neonatal mortality and suspected maternal infection. 49 This information needs to somehow be incorporated into the review to maintain its credibility.

A Cochrane review of community deworming in developing countries indicates that in recent studies, there is little or no effect. 50 The inclusion of a large trial of two million children confirmed that there was no effect on mortality. Although the incorporation of the trial in the review did not change the review’s conclusions, the trial’s absence would have affected the credibility of the review, so it was therefore updated.

A new paper reporting long term follow-up data on anthracycline chemotherapy as part of cancer treatment was published. Although the effects from the outcomes remained essentially unchanged, apart from this longer follow-up, the paper also included information about the performance bias in the original trial, shifting the risk of bias for several outcomes from “unknown” to “high” in the Cochrane review. 51

Reviews with a high level of certainty in the results (that is, when the GRADE assessment for the body of evidence is high) are less likely to change even with the addition of new studies, information, or data, by definition. GRADE can help guide priorities in whether to update, but it is still important to assess new studies that might meet the inclusion criteria. New studies can show unexpected effects (eg, attenuation of efficacy) or provide new information about the effects seen in different circumstances (eg, groups of patients or locations).

Other tools are specifically designed to help decision making in updating. For example, the Ottawa 39 and RAND 45 methods focus on identification of new evidence, the statistical predication tool 15 calculates the probability of new evidence changing the review conclusion, and the value of information analysis approach 52 calculates the expected health gain (table 1 ⇑ ). As yet, there has been limited external validation of these tools to determine which approach would be most effective and when.

If potentially relevant studies are identified that have not previously been assessed for inclusion, authors or those managing the updating process need to assess whether including them might affect the conclusions of the review. They need to examine the weight and certainty of the new evidence to help determine whether an update is needed and how urgent that update is. The updating team can assess this informally by judging whether new studies or data are likely to substantively affect the review, for example, by altering the certainty in an existing comparison, or by generating new comparisons and analyses in the existing review.

New information can also include fresh follow-up data on existing included studies, or information on how the studies were carried out. These should be assessed in terms of whether they might change the review findings or improve its credibility (fig 1 ⇑ ). Indeed, if any study has been retracted, it is important the authors assess the reasons for its retraction. In the case of data fabrication, the study needs to be removed from the analysis and this recorded. A decision needs to be made as to whether other studies by the same author should be removed from the review and other related reviews. An investigation should also be initiated following guidelines from the Committee on Publication Ethics (COPE). Additional published and unpublished data can become available from a wide range of sources—including study investigators, regulatory agencies and industry—and are important to consider.

Preparing for an update

Refresh background, objectives, inclusion criteria, and methods

Before including new studies in the review, authors need to revisit the background, objectives, inclusion criteria, and methods of the current review. In Cochrane, this is referred to as the protocol, and editors are part of this process. The update could range from simply endorsing the current question and inclusion criteria, through to full rewriting of the question, inclusion criteria and methods, and republishing the protocol. As a field progresses with larger and better quality trials rigorously testing the questions posed, it may be appropriate to exclude weaker study designs (such as quasi-randomised comparisons or very small trials) from the update (table 2 ⇓ ). The PUGs panel recommended that a protocol refresh will require the authors to use the latest accepted methods of synthesis, even if this means repeating data extraction for all studies.

New authors and authorship

Updated systematic reviews are new publications with new citations. An authorship team publishing an update in a scientific or medical journal is likely to manage the new edition of a review in the same way as with any other publication, and follow the ICMJE authorship criteria. 56 If the previous author or author team steps down, then they should be acknowledged in the new version. However, some might perceive that their efforts in the first version warrant continued authorship, which may be valid. The management of authorship between versions can sometimes be complicated. At worst, it delays new authors completing an update and leads to long authorship lists of people from previous versions who probably do not meet ICMJE authorship criteria. One approach with updates including new authors is to have an opt-in policy for the existing authors: they can opt in to the new edition, provided that they make clear their contribution, and this is then agreed with the entire author team.

Although they are new publications, updates will generally include content from the published version. Changing licensing rights around systematic reviews to allow new authors of future updates to remix, tweak, or build on the contributions of the original authors of the published version (similar to the rights available via a Creative Commons licence; https://creativecommons.org ) could be a more sustainable and simpler approach. This approach would allow systematic reviews to continue to evolve and build on the work of a range of authors over time, and for contributors to be given credit for contributions to this previous work.

Efficient searching

In performing an update, a search based on the search conducted for the original review is required. The updated search strategy will need to take into account changes in the review question or inclusion criteria, for example, and might be further adjusted based on knowledge of running the original search strategy. The search strategy for an update need not replicate the original search strategy, but could be refined, for example, based on an analysis of the yield of the original search. These new search approaches are currently undergoing formal empirical evaluation, but they may well provide much more efficient search strategies in the future. Some examples of these possible new methods for review updates are described in web appendix 2.

In reporting the search process for the update, investigators must ensure transparency for any previous versions and the current update, and use an adapted flow diagram based on PRISMA reporting (preferred reporting items for systematic reviews and meta-analyses). 57 The search processes and strategies for the update must be adequately reported such that they could be replicated.

Systematic reviews published for the first time in peer reviewed journals are by definition peer reviewed, but practice for updates remains variable, because an update might have few changes (such as an updated search but no new studies found and therefore included) or many changes (such as revise methods and inclusion of several new studies leading to revised conclusions). Therefore, and to use peer reviewers’ time most effectively, editors need to consider when to peer review an update and the type of peer reviewer most useful for a particular update (for example, topic specialist, methodologist). The decision to use peer review, and the number and expertise of the peer reviewers could depend on the nature of the update and the extent of any changes to the systematic review as part of an editor assessment. A change in the date of the search only (where no new studies were identified) would not require peer review (except, arguably, peer review of the search), but the addition of studies that lead to a change in conclusions or significant changes to the methods would require peer review. The nature of the peer review could be described within the published article.

Reporting changes

Authors should provide a clear description of the changes in approach or methods between different editions of a review. Also, authors need to report the differences in findings between the original and updated edition to help users decide how to use the new edition. The approach or format used to present the differences in findings might vary with the target user group. 58 Publishers need to ensure that all previous versions of the review remain publically accessible.

Updates can range from small adjustments to reviews being completely rewritten, and the PUGs panel spent some time debating whether the term “new edition” would be a better description than “update.” However, the word “update” is now in common parlance and changing the term, the panel judged, could cause confusion. However, the debate does illustrate that an update could represent a review that asks a similar question but has been completely revised.

Technology and innovation

The updating of systematic review is generally done manually and is time consuming. There are opportunities to make better use of technology to streamline the updating process and improve efficiency (table 3 ⇓ ). Some of these tools already exist and are in development or in early use, and some are commercially available or freely available. The AHRQ’s evidence based practice centre team has recently published tools for searching and screening, and will provide an assessment of the use, reliability, and availability of these tools. 63

Technological innovations to improve the efficiency of updating systematic reviews

Other developments, such as targeted updates that are performed rapidly and focus on updating only key components of a review, could provide different approaches to updating in the future and are being piloted and evaluated. 64 With implementation of these various innovations, the longer term goal is for “living” systematic reviews, which identify and incorporate information rapidly as it evolves over time. 60

Concluding remarks

Updating systematic reviews, rather than addressing the same question with a fresh protocol, is generally more efficient and allows incremental improvement over time. Mechanical rules appear unworkable, but there is no clear unified approach on when to update, and how implement this. This PUGs panel of authors, editors, statisticians, information specialists, other methodologists, and guideline developers brought together current thinking and experience in this area to provide guidance.

Decisions about whether and when to update a systematic review are judgments made at a point in time. They depend on the currency of the question asked, the need for updating to maintain credibility, the availability of new evidence, and whether new research or new methods will affect the findings.

Whether the review uses current methodological standards is important in deciding if the update will influence the review findings, quality, reliability, or credibility sufficiently to justify the effort in updating it. Those updating systematic reviews to author clinical practice guidelines might consider the influence of new study results in potentially overturning the conclusions of an existing review. Yet, even in cases where new study findings do not change the primary outcome measure, new studies can carry important information about subgroup effects, duration of treatment effects, and other relevant clinical information, enhancing the currency and breadth of review results.

An update requires appraisal and revision of the background, question, inclusion criteria, and methods of the existing review and the existing certainty in the evidence. In particular, methods might need to be updated, and search strategies reconsidered. Authors of updates need to consider inputs to the current edition, and follow ICMJE criteria regarding authorship. 56

The PUGs panel proposed a decision framework (fig 1 ⇑ ), with terms and categories for reporting the decisions made for updating procedures for adoption by Cochrane and other stakeholders. This framework includes journals publishing systematic review updates and independent authors considering updates of existing published reviews. The panel developed a checklist to help judgements about when and how to update.

The current emphasis of authors, guideline developers, Cochrane, and consequently this guidance has been on effects reviews. The checklists and guidance here still applies to other types of systematic reviews, such as those on diagnostic test accuracy, and this guidance will need adapting. Accumulative experience and methods development in reviews other than those of effects are likely to help refine guidance in the future.

This guidance could help groups identify and prioritise reviews for updating and hence use their finite resources to greatest effect. Software innovation and new management systems are being developed and in early use to help streamline review updates in the coming years.

Contributors: HJS initiated the workshop. JC, SH, PG, HM, and HJS organised the materials and the agenda. SH wrote up the proceedings. PG wrote the paper from the proceedings and coordinated the development of the final guidance; JC, SH, HM, and HJS were active in the finalising of the guidance. All PUGs authors contributed to three rounds of manuscript revision.

Funding: Attendance at this meeting, for those attendees not directly employed by Cochrane, was not funded by Cochrane beyond the reimbursement of out of pocket expenses for those attendees for whom this was appropriate. Expenses were not reimbursed for US federal government attendees, in line with US government policy. Statements in the manuscript should not be construed as endorsement by the US Agency for Healthcare Research and Quality or the US Department of Health and Human Services.

Competing interests: All participants have a direct or indirect interest in systematic reviews and updating as part of their job or academic career. Most participants contribute to Cochrane, whose mission includes a commitment to the updating of its systematic review portfolio. JC, HM, RM, CM, KS-W, and MT are, or were at that time, employed by the Cochrane Central Executive.

Provenance and peer review: Not commissioned; externally peer reviewed.

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 3.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/3.0/ .

  • ↵ Shekelle PG, Ortiz E, Rhodes S, et al. Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA 2001 ; 286 : 1461 - 7 . doi:10.1001/jama.286.12.1461 pmid:11572738 . OpenUrl CrossRef PubMed Web of Science
  • ↵ Claxton K, Cohen JT, Neumann PJ. When is evidence sufficient? Health Aff (Millwood) 2005 ; 24 : 93 - 101 . doi:10.1377/hlthaff.24.1.93 pmid:15647219 . OpenUrl Abstract / FREE Full Text
  • ↵ Fenwick E, Claxton K, Sculpher M, et al. Improving the efficiency and relevance of health technology assessment: the role of decision analytic modelling. Paper 179. Centre for Health Economics, University of York, 2000 .
  • ↵ Sculpher M, Claxton K. Establishing the cost-effectiveness of new pharmaceuticals under conditions of uncertainty—when is there sufficient evidence? Value Health 2005 ; 8 : 433 - 46 . doi:10.1111/j.1524-4733.2005.00033.x pmid:16091019 . OpenUrl CrossRef PubMed Web of Science
  • ↵ Sculpher M, Drummond M, Buxton M. The iterative use of economic evaluation as part of the process of health technology assessment. J Health Serv Res Policy 1997 ; 2 : 26 - 30 . pmid:10180650 . OpenUrl Abstract / FREE Full Text
  • ↵ Wilson E, Abrams K. From evidence based economics to economics based evidence: using systematic review to inform the design of future research. In: Shemilt I, Mugford M, Vale L, et al, eds. Evidence based economics. Blackwell Publishing, 2010 doi:10.1002/9781444320398.ch12 .
  • ↵ Chalmers I, Enkin M, Keirse MJ. Preparing and updating systematic reviews of randomized controlled trials of health care. Milbank Q 1993 ; 71 : 411 - 37 . doi:10.2307/3350409 pmid:8413069 . OpenUrl CrossRef PubMed Web of Science
  • ↵ Higgins J, Green S, Scholten R. Chapter 3. Maintaining reviews: updates, amendments and feedback: Version 5.1.0 (updated March 2011). Cochrane Collaboration, 2011 .
  • ↵ Cochrane. Editorial and publishing policy resource. http://community.cochrane.org/editorial-and-publishing-policy-resource . 2016.
  • ↵ Moher D, Tsertsvadze A. Systematic reviews: when is an update an update? Lancet 2006 ; 367 : 881 - 3 . doi:10.1016/S0140-6736(06)68358-X pmid:16546523 . OpenUrl CrossRef PubMed Web of Science
  • ↵ Prasad K, Singh MB, Ryan H. Corticosteroids for managing tuberculous meningitis. Cochrane Database Syst Rev 2016 ; 4 : CD002244 . pmid:27121755 . OpenUrl PubMed
  • ↵ Zani B, Gathu M, Donegan S, Olliaro PL, Sinclair D. Dihydroartemisinin-piperaquine for treating uncomplicated Plasmodium falciparum malaria. Cochrane Database Syst Rev 2014 ; 1 : CD010927 . pmid:24443033 . OpenUrl CrossRef PubMed
  • ↵ Adams SP, Tsang M, Wright JM. Lipid lowering efficacy of atorvastatin. Cochrane Database Syst Rev 2012 ; 12 : CD008226 . pmid:23235655 . OpenUrl PubMed
  • ↵ Higgins J. Convincing evidence from controlled and uncontrolled studies on the lipid-lowering effect of a statin. Cochrane Database Syst Rev 2012 ; 12 : ED000049 . pmid:23361645 . OpenUrl PubMed
  • ↵ Takwoingi Y, Hopewell S, Tovey D, Sutton AJ. A multicomponent decision tool for prioritising the updating of systematic reviews. BMJ 2013 ; 347 : f7191 . doi:10.1136/bmj.f7191 pmid:24336453 . OpenUrl FREE Full Text
  • ↵ MacLehose H, Hilton J, Tovey D, et al. The Cochrane Library: revolution or evolution? Shaping the future of Cochrane content. Background paper for The Cochrane Collaboration’s Strategic Session Paris, France, 18 April 2012. http://editorial-unit.cochrane.org/sites/editorial-unit.cochrane.org/files/uploads/2012-CC-strategic-session_full-report.pdf .
  • ↵ Hemilä H, Chalker E. Vitamin C for preventing and treating the common cold. Cochrane Database Syst Rev 2013 ;( 1 ): CD000980 . pmid:23440782 .
  • ↵ Hahn S, Kim S, Garner P. Reduced osmolarity oral rehydration solution for treating dehydration caused by acute diarrhoea in children. Cochrane Database Syst Rev 2002 ;( 1 ): CD002847 . pmid:11869639 .
  • ↵ World Health Organization (WHO). Reduced osmolarity oral rehydration salts (ORS) formulation. A report from a meeting of Experts jointly organized by UNICEF and WHO. New York: Child and Adolescent Health and Development, 18 July 2001 http://apps.who.int/iris/bitstream/10665/67322/1/WHO_FCH_CAH_01.22.pdf .
  • ↵ Smaill FM, Grivell RM. Antibiotic prophylaxis versus no prophylaxis for preventing infection after cesarean section. Cochrane Database Syst Rev 2014 ;( 10 ): CD007482 . pmid:25350672 .
  • ↵ Adjuik M, Babiker A, Garner P, Olliaro P, Taylor W, White N. International Artemisinin Study Group. Artesunate combinations for treatment of malaria: meta-analysis. Lancet 2004 ; 363 : 9 - 17 . doi:10.1016/S0140-6736(03)15162-8 pmid:14723987 . OpenUrl CrossRef PubMed Web of Science
  • ↵ Agnihotry A, Fedorowicz Z, Nasser M. Adhesively bonded versus non-bonded amalgam restorations for dental caries. Cochrane Database Syst Rev 2016 ; 3 : CD007517 . pmid:26954446 . OpenUrl PubMed
  • ↵ Higgins JP, Altman DG, Gøtzsche PC, et al. Cochrane Bias Methods Group Cochrane Statistical Methods Group. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011 ; 343 : d5928 . doi:10.1136/bmj.d5928 pmid:22008217 . OpenUrl FREE Full Text
  • ↵ Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011 ; 155 : 529 - 36 . doi:10.7326/0003-4819-155-8-201110180-00009 pmid:22007046 . OpenUrl CrossRef PubMed Web of Science
  • ↵ Sterne JAC, Higgins JPT, Reeves BC; on behalf of the development group for ROBINS-I. A tool for assessing risk of bias in non-randomized studies of interventions, version 7. March 2016. www.riskofbias.info .
  • ↵ Guyatt GH, Oxman AD, Vist GE, et al. GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008 ; 336 : 924 - 6 . doi:10.1136/bmj.39489.470347.AD pmid:18436948 . OpenUrl FREE Full Text
  • ↵ Schünemann HJ. Interpreting GRADE’s levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision? J Clin Epidemiol 2016 ; 75 : 6 - 15 . doi:10.1016/j.jclinepi.2016.03.018 pmid:27063205 . OpenUrl CrossRef PubMed
  • ↵ Gough D. Qualitative and mixed methods in systematic reviews. Syst Rev 2015 ; 4 : 181 . doi:10.1186/s13643-015-0151-y pmid:26670769 . OpenUrl CrossRef PubMed
  • ↵ Richardson M, Garner P, Donegan S. Cluster randomised trials in Cochrane reviews: evaluation of methodological and reporting practice. PLoS One 2016 ; 11 : e0151818 . doi:10.1371/journal.pone.0151818 pmid:26982697 . OpenUrl CrossRef PubMed
  • ↵ Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007 ; 7 : 10 . doi:10.1186/1471-2288-7-10 pmid:17302989 . OpenUrl CrossRef PubMed
  • ↵ Whiting P, Savović J, Higgins JP, et al. ROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016 ; 69 : 225 - 34 . doi:10.1016/j.jclinepi.2015.06.005 pmid:26092286 . OpenUrl CrossRef PubMed
  • ↵ Sampson M, Shojania KG, McGowan J, et al. Surveillance search techniques identified the need to update systematic reviews. J Clin Epidemiol 2008 ; 61 : 755 - 62 . doi:10.1016/j.jclinepi.2007.10.003 pmid:18586179 . OpenUrl CrossRef PubMed Web of Science
  • ↵ Hemens BJ, Haynes RB. McMaster Premium LiteratUre Service (PLUS) performed well for identifying new studies for updated Cochrane reviews. J Clin Epidemiol 2012 ; 65 : 62 - 72.e1 . doi:10.1016/j.jclinepi.2011.02.010 pmid:21856121 . OpenUrl CrossRef PubMed
  • ↵ Sagliocca L, De Masi S, Ferrigno L, Mele A, Traversa G. A pragmatic strategy for the review of clinical evidence. J Eval Clin Pract 2013 ; 19 : 689 - 96 . doi:10.1111/jep.12020 pmid:23317014 . OpenUrl CrossRef PubMed
  • ↵ Rada G, Peña J, Capurro D, et al. How to create a matrix of evidence in Epistemonikos. Abstracts of the 22nd Cochrane Colloquium; Evidence-informed public health: opportunities and challenges; Hyderabad, India. Cochrane Database Syst Rev 2014 ; suppl 1 : 132 .
  • ↵ Okebe JU, Yahav D, Shbita R, Paul M. Oral iron supplements for children in malaria-endemic areas. Cochrane Database Syst Rev 2011 ;( 10 ): CD006589 . pmid:21975754 .
  • ↵ Neuberger A, Okebe J, Yahav D, Paul M. Oral iron supplements for children in malaria-endemic areas. Cochrane Database Syst Rev 2016 ; 2 : CD006589 . pmid:26921618 . OpenUrl PubMed
  • Balshem H, Helfand M, Schünemann HJ, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011 ; 64 : 401 - 6 . doi:10.1016/j.jclinepi.2010.07.015 pmid:21208779 . OpenUrl CrossRef PubMed Web of Science
  • ↵ Chung M, Newberry SJ, Ansari MT, et al. Two methods provide similar signals for the need to update systematic reviews. J Clin Epidemiol 2012 ; 65 : 660 - 8 . doi:10.1016/j.jclinepi.2011.12.004 pmid:22464414 . OpenUrl CrossRef PubMed
  • Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 2007 ; 147 : 224 - 33 . doi:10.7326/0003-4819-147-4-200708210-00179 pmid:17638714 . OpenUrl CrossRef PubMed Web of Science
  • Shojania K, Sampson M, Ansari M, et al. Updating systematic reviews; AHRQ technical reviews; report no 07-0087. Agency for Healthcare Research and Quality, 2007 .
  • Pattanittum P, Laopaiboon M, Moher D, Lumbiganon P, Ngamjarus C. A comparison of statistical methods for identifying out-of-date systematic reviews. PLoS One 2012 ; 7 : e48894 . doi:10.1371/journal.pone.0048894 pmid:23185281 . OpenUrl CrossRef PubMed
  • Shekelle PG, Motala A, Johnsen B, Newberry SJ. Assessment of a method to detect signals for updating systematic reviews. Syst Rev 2014 ; 3 : 13 . doi:10.1186/2046-4053-3-13 pmid:24529068 . OpenUrl CrossRef PubMed
  • Shekelle PG, Newberry SJ, Wu H, et al. Identifying signals for updating systematic reviews: a comparison of two methods; report no 11-EHC042-EF. Agency for Healthcare Research and Quality, 2011 .
  • ↵ Shekelle P, Newberry S, Maglione M, et al. Assessment of the need to update comparative effectiveness reviews: report of an initial rapid program assessment (2005-2009). Agency for Healthcare Research and Quality, 2009 .
  • Tovey D, Marshall R, Bazian, Hopewell S, Rader T. Fit for purpose: centralised updating support for high-priority Cochrane Reviews; National Institute for Health Research Cochrane-National Health Service Engagement Award Scheme, July 2011. https://editorial-unit.cochrane.org/sites/editorial-unit.cochrane.org/files/uploads/10_4000_01%20Fit%20for%20purpose%20-%20centralised%20updating%20support%20for%20high%20priority%20Cochrane%20Reviews%20FINAL%20REPORT.pdf .
  • Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ 1999 ; 18 : 341 - 64 . doi:10.1016/S0167-6296(98)00039-3 pmid:10537899 . OpenUrl CrossRef PubMed Web of Science
  • Wilson EC. A practical guide to value of information analysis. Pharmacoeconomics 2015 ; 33 : 105 - 21 . doi:10.1007/s40273-014-0219-x pmid:25336432 . OpenUrl CrossRef PubMed
  • ↵ Althabe F, Belizán JM, McClure EM, et al. A population-based, multifaceted strategy to implement antenatal corticosteroid treatment versus standard care for the reduction of neonatal mortality due to preterm birth in low-income and middle-income countries: the ACT cluster-randomised trial. Lancet 2015 ; 385 : 629 - 39 . doi:10.1016/S0140-6736(14)61651-2 pmid:25458726 . OpenUrl CrossRef PubMed
  • ↵ Taylor-Robinson D, Maayan N, Soares-Weiser K, et al. Deworming drugs for soil-transmitted intestinal worms in children: effects on nutritional indicators, haemoglobin and school performance. Cochrane Database Syst Rev 2012 ;( 11 ): CD000371 .
  • ↵ van Dalen EC, van der Pal HJ, Kremer LC. Different dosage schedules for reducing cardiotoxicity in people with cancer receiving anthracycline chemotherapy. Cochrane Database Syst Rev 2016 ; 3 : CD005008 . pmid:26938118 . OpenUrl PubMed
  • ↵ Wilson E. on behalf of the Cochrane Priority Setting and Campbell & Cochrane Economics Methods Groups. Which study when? Proof of concept of a proposed automated tool to help decision which reviews to update first. Cochrane Database Syst Rev 2014 ; suppl 2 : 29 - 31 .
  • Rosenbaum SE, Glenton C, Nylund HK, Oxman AD. User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. J Clin Epidemiol 2010 ; 63 : 607 - 19 . doi:10.1016/j.jclinepi.2009.12.013 pmid:20434023 . OpenUrl CrossRef PubMed
  • Rosenbaum SE, Glenton C, Oxman AD. Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. J Clin Epidemiol 2010 ; 63 : 620 - 6 . doi:10.1016/j.jclinepi.2009.12.014 pmid:20434024 . OpenUrl CrossRef PubMed Web of Science
  • Vandvik PO, Santesso N, Akl EA, et al. Formatting modifications in GRADE evidence profiles improved guideline panelists comprehension and accessibility to information. A randomized trial. J Clin Epidemiol 2012 ; 65 : 748 - 55 . doi:10.1016/j.jclinepi.2011.11.013 pmid:22564503 . OpenUrl CrossRef PubMed
  • ↵ International Committee of Medical Journal Editors (ICMJE). Defining the role of authors and contributors. 2016. www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html .
  • ↵ Stovold E, Beecher D, Foxlee R, Noel-Storr A. Study flow diagrams in Cochrane systematic review updates: an adapted PRISMA flow diagram. Syst Rev 2014 ; 3 : 54 . doi:10.1186/2046-4053-3-54 pmid:24886533 . OpenUrl CrossRef PubMed
  • ↵ Newberry SJ, Shekelle PG, Vaiana M, et al. Reporting the findings of updated systematic reviews of comparative effectiveness: how do users want to view new information? report no 13-EHC093-EF. Agency for Healthcare Research and Quality, 2013 .
  • Marshall IJ, Kuiper J, Wallace BC. Automating risk of bias assessment for clinical trials BCB’14. Proceedings of the 5th ACM conference on Bioinformatics, computational biology, and health informatics. 2014:88-95. http://thirdworld.nl/automating-risk-of-bias-assessment-for-clinical-trials .
  • ↵ Elliott JH, Turner T, Clavisi O, et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med 2014 ; 11 : e1001603 . doi:10.1371/journal.pmed.1001603 pmid:24558353 . OpenUrl CrossRef PubMed
  • Elliott J, Sim I, Thomas J, et al. #CochraneTech: technology and the future of systematic reviews. Cochrane Database Syst Rev 2014 ;( 9 ): ED000091 . pmid:25288182 .
  • Cochrane. Project transform: the Cochrane Collaboration. 2016. http://community.cochrane.org/tools/project-coordination-and-support/transform .
  • ↵ Paynter R, Bañez L, Berlinerm E, et al. EPC methods: an exploration of the use of text-mining software in systematic reviews. Research white paper. AHRQ publication 16-EHC023-EF. Agency for Healthcare Research and Quality, April 2016. https://www.effectivehealthcare.ahrq.gov/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productID=2214 .
  • ↵ Soares-Weiser K, Marshall R, Bergman H, et al. Updating Cochrane Reviews: results of the first pilot of a focused update. Cochrane Database Syst Rev 2014 ; suppl 1 : 31 - 3 .

an updated review of the literature

Jump to navigation

Home

Cochrane Training

Chapter iv: updating a review.

Miranda Cumpston and Ella Flemyng

Key Points:

  • As new studies are completed, the results of reviews may become out of date and thereby provide misleading information to decision makers.
  • Cochrane Reviews should be assessed periodically to determine whether an update is needed. The decision to update should be based on the continuing importance of the review question to decision makers and the availability of new data or new methods that would have a meaningful impact on the review findings.
  • A review update provides an opportunity for the scope, eligibility criteria and methods used in the review to be revised.
  • An update should be conducted according to the standards required for any review, with some additional requirements to ensure that any changes are managed appropriately and reported clearly to readers.

This chapter should be cited as: Cumpston M, Flemyng E. Chapter IV: Updating a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook .

IV.1 Introduction

Since its inception, Cochrane has sought to maintain its reviews to ensure they are updated to include the most recent evidence. Reviews that are out of date and do not incorporate all the available evidence risk providing misleading information to decision makers and other stakeholders.

Garner and colleagues define an update as “a new edition of a published systematic review with changes that can include new data, new methods, or new analyses to the previous edition” (Garner et al 2016). Adding new studies and new data can substantively change the findings of the review. Even where the new studies observe results consistent with the existing data, increasing the number of studies can improve precision of effect estimates, demonstrate wider applicability of the effect, or enable additional comparisons or subgroup analyses to be performed. The introduction of new review methods, such as updated risk of bias assessment tools or improved statistical analysis methods, can also change both the results and the certainty of the review’s findings. Examples of the impact of incorporating new information and methods are illustrated in Box IV.1.a .

All Cochrane Reviews should be assessed periodically to determine whether an update is needed. Some areas of research evolve rapidly, whereas others are more stable, and some research questions stop being relevant to decision makers. A report assessing 100 systematic reviews published between 1995 and 2005 concluded the median time to require an update was 5.5 years, although 23% of reviews were out of date within two years, 15% within one year, and 7% were already out of date at the time of publication (Shojania et al 2007). Authors of Cochrane Reviews should therefore consider both whether an update is warranted, and when it will be most beneficial for each specific review (see Section IV.2 ).

In some areas, authors are establishing ‘living’ systematic reviews that adopt a continual updating process, such as monthly searching followed by rapid incorporation of new evidence into the published review. Living systematic reviews are most likely to be appropriate for questions that are of high importance to decision makers, and for which new evidence is likely to be frequently published and to have an important impact on the review’s findings (Elliott et al 2017). Considerable resources are required to support such an ongoing process. Further discussion of living systematic reviews is presented in Chapter 22, Section 22.2.3 .

Cochrane’s Methodological Expectations of Cochrane Intervention Reviews (MECIR) , which guide the conduct of Cochrane Reviews, include expectations for updating reviews. See the online MECIR Manual for the 11 expectations specifically relevant to updates, although updated reviews should also meet the expectations that apply to all reviews. This chapter elaborates on the recommendations for the planning, conduct and reporting of Cochrane Review updates.

Box IV.1.a Examples of what factors might change in an updated systematic review (Garner et al 2016). Reproduced from Garner P, Hopewell S, Chandler J, MacLehose H, Akl EA, Beyene J, et al. When and how to update systematic reviews: consensus and checklist. BMJ 2016; 354: i3507 licensed under CC BY 3.0 .

IV.2 Deciding whether and when to update

The decision to undertake an update of a review requires consideration of a number of different factors. Garner and colleagues conducted an international consensus process to establish good practice guidance for determining when a systematic review should be updated (Garner et al 2016). Their published framework and checklist can assist authors in thinking through these issues in a structured way (see Figure IV.2.a ).

Figure IV.2.a Decision framework to assess systematic reviews for updating, with standard terms to report such decisions (Garner et al 2016). Reproduced from Garner P, Hopewell S, Chandler J, MacLehose H, Akl EA, Beyene J, et al. When and how to update systematic reviews: consensus and checklist. BMJ 2016; 354: i3507 licensed under CC BY 3.0 .

an updated review of the literature

When deciding whether to update a particular review, the first consideration should be to determine whether the review question remains relevant to decision makers, and is well-targeted to answer current questions in policy and practice. Knowledge of the particular field will be required to answer this question. Checking whether the existing review is frequently accessed or cited can also be useful to indicate whether there is a need to update. A second aspect to this question is whether the original review was conducted well and used appropriate methods (Garner et al 2016). If the review question remains fundamentally of interest, additions and improvements may be possible to enhance the review’s methods (see Section IV.3.4 ). Depending on the changes required, it may be more appropriate to conduct a new review from scratch meeting current standards. A comparison between currently recommended methods and the methods used in the review can identify any important changes required.

If the review remains important and is of a sufficient standard, then the next step is to consider whether there are any new studies, newly available information, or newly recommended methods that could be incorporated into the review. The existing version of the review may include details of ongoing studies identified at the time of its publication, for example through searches of trials registers, and these trials may now be complete. Some authors may choose to monitor the literature continually for new studies (e.g. through automated alerts), or may conduct a rapid scoping search for this purpose.

If either new information or new methodology is available, a critical next step is to evaluate whether incorporating these into the review would be likely to impact on its findings (Garner et al 2016). In some cases, this decision can be very straightforward, for example when the existing reviews findings are considered very uncertain (for example, using the GRADE approach to assessment, see Chapter 14 ). For some reviews, the findings are of very high certainty, and it is unlikely that new information will meaningfully impact the conclusions. In some cases, maintaining credibility through the incorporation of additional information and new methods is sufficient in itself to warrant updating (Garner et al 2016).

In some cases, although the main findings of the review may be unaffected, additional information may shed light on more nuanced effects of different variations on the intervention, different settings, additional outcomes, or population subgroups. In other cases, it may not be clear whether the extent of new information available will be enough to impact meaningfully on the results (Garner et al 2016).

To date there is no consensus on when to update a review (Tsertsvadze et al 2011), although several methods have been proposed (e.g. Sampson et al (2008), Shekelle et al (2011), Tovey et al (2011), Ahmadzai et al (2013), Takwoingi et al (2013)). These methods use signals to indicate the need for an update and the likely impact of new studies on existing conclusions. They include surveillance searches, contact with experts, and quantitative or qualitative assessments, or both. Chapter 22, Section 22.2 , outlines a range of methods for surveillance of the literature and the interpretation of signals for updating, including statistical methods based on sample size calculations or the application of prediction equations to assess the impact of new evidence. Garner and colleagues also summarize a series of available methods (Garner et al 2016). Ultimately, review authors should make a judgement based on an individual assessment and their knowledge of the field covered by the review.

IV.3 Planning an update

Before embarking on an updated review, it is important to take the time to plan the process. Any proposed modifications or additions to the existing review should be planned in detail, and on occasion may require drafting a new protocol for the review. In addition, there are several issues unique to updates that should be considered.

Many of the approaches using new technologies designed to facilitate the review process are intended to support easier and more frequent updates. Further information is available in Chapter 4 , Section 4.6.6 , and Chapter 22 , Section 22.2.4 .

See the online MECIR Manual for expectations relevant to planning an update.

IV.3.1 Reconsidering review questions and eligibility criteria

Even when the overall review question has been agreed to remain relevant, an update is an opportunity to consider changes to the question and its scope. Authors should reconsider all elements of the review question (PICO), the eligibility criteria, comparisons and outcomes of interest. For example, evolving understanding of the problem may lead to the inclusion of a new comparison, an additional category of patients (e.g. children in addition to adults) or an important new outcome (e.g. adverse effects) that may not have been adequately addressed in the original review. Review authors may also wish to include additional objectives, such as addressing the economic aspects of the intervention or its implementation. Additional engagement with stakeholders may reveal current issues around which there is uncertainty (see Chapter 2 ).

Irrespective of whether the review question(s) change, there may be reason to amend the eligibility criteria for the review (see Chapter 3 ). For example, new intervention options may have become available since the publication of the original review. As the number of available studies increases over time, this may also affect decisions about eligibility. For example, if the original review included both randomized trials and non-randomized studies, and the former provide sufficient evidence to answer the review questions, it may be reasonable to decide to exclude non-randomized studies from subsequent updates of the review. Conversely, it may be reasonable to add non-randomized studies to a review that was previously restricted to randomized trials, to widen the evidence base, making use of methodological developments in critical evaluation of the validity of non-randomized studies (see Chapter 24 ).

IV.3.2 Splitting and merging reviews

As the body of evidence accumulates over time, a review may become too large for authors to manage (some of the largest Cochrane Reviews include hundreds of studies across multiple comparisons). It is sometimes appropriate to consider splitting the review into two or more reviews with more narrowly defined questions. For example, an early Cochrane Review investigated all interventions for shoulder pain. As this review became large and unwieldy over time, it was split into multiple separate reviews, each looking at an intervention category. One of these reviews looked at physiotherapy interventions for shoulder pain (Green et al 2003). As time went on, this review also became too large to manage, and was split into a number of reviews examining different physiotherapy interventions and specific types of shoulder pain (e.g. Page et al (2014a), Page et al (2014b), Page et al (2016a), Page et al (2016b)).

Narrower reviews may allow deeper investigation of specific intervention types, and more focused information for stakeholders, and may distribute the updating burden between several review author teams. On the other hand, narrower reviews can sometimes prevent readers from considering findings across all the interventions relevant to a decision (see Chapter 2, Section 2.3 ). Overviews of Reviews are an alternative option, allowing authors to summarize several more narrowly defined reviews that may have been split from a larger review (see Chapter V ).

It is also possible for one or more narrower Cochrane Reviews to be merged into a larger review, where agreed by all authors that this would present a more useful synthesis for decision makers. For example, it might be concluded that a network meta-analysis to compare multiple intervention options for a particular condition would be more useful than an existing series of separate reviews of specific interventions (see Chapter 11 ).

IV.3.3 Planning the search strategy for an update

Once the scope and eligibility criteria for the update have been agreed, authors will prepare for an update by deciding on the appropriate search process and strategy.

A starting point for identifying new studies for inclusion may be those already identified as ongoing studies at the time of the existing version of the review. Following this, in some cases, the search strategy can be re-run as specified in the existing review, with the addition of date limits set to the period following the most recent search. However, an information specialist or healthcare librarian should be consulted to ensure the strategy remains appropriate. Changes to electronic databases, their access mechanisms and controlled vocabulary can require expert amendments to the search strategies. In addition, informed by the experience of the search for the original review, a decision may be made to modify the list of sources to be searched or search terms to be used (Garner et al 2016).

If important changes to the PICO for the review or the eligibility criteria have been made since the original search, or developments in the field have led to the emergence of new terms to be added to the search, it may be necessary to re-run parts of the search back to the earliest records, to ensure that any records relevant to new search terms were not missed in the original search.

IV.3.4 Planning the methods for an update

Methodological advances in systematic review conduct since publication of the original review may result in a need to revise or extend the methods of the review update (Shea et al 2006). Authors are encouraged to consult current guidance on review methods and compare these with the methods used in the existing review to identify important changes.

Examples of situations in which review methodology might be updated include:

  • incorporating updated guidance on risk of bias assessment (see Chapter 7 and Chapter 8 );
  • using a new synthesis strategy, such as an improved method to perform a random-effects meta-analysis (see Chapter 10 ), or alternative methods for synthesis where meta-analysis is not possible (see Chapter 12 ).
  • incorporating GRADE assessments and ‘Summary of findings’ tables if not already included (see Chapter 14 ); and
  • adopting new guidance on the structure and presentation of findings, such as structured tabulation of results or alternative methods for visual presentation of results in reviews where meta-analysis is not used (see Chapter 12 ).

Changes to the scope of the review, such as expansion to include different study designs or outcome data, will require planning for new methods appropriate to the data expected.

Where changes to the review methods are substantive, authors are encouraged to write a complete, updated protocol to guide the conduct of the review update . In some cases, it may be more appropriate to consider the work as a new review, rather than an update.

Specific methods developed for systematic reviews that conduct ongoing and prospective approaches to accumulating evidence to maintain review currency are outlined in Chapter 22 . Formal sequential statistical methods that aim to address errors associated with repeating meta-analyses over time have been developed. However, such approaches are explicitly discouraged for updated meta-analyses in Cochrane Reviews, except in the context of a prospectively planned series of primary research studies (see Chapter 22, Section 22.4 ).

IV.3.5 Incorporating feedback and comments

Updating a published review provides an opportunity to consider any feedback or comments submitted to Cochrane or directly to the authors. Review authors are expected to be responsive to comments on their reviews, in the spirit of the scientific process and publication ethics. Comments may represent valid concerns and can usefully identify additional studies that were overlooked by the review authors.

IV.4 Conducting an update

An update of a review should be conducted according to the protocol, as closely as possible to the methods of the existing review while incorporating any planned changes (see Section IV.3 ). All steps should be conducted in accordance with the guidance presented throughout this Handbook .

A systematic search should be conducted for new studies (see Chapter 4 ), and the date of the search should be within 12 months of publication of the update. If new, potentially relevant studies are found, they should be assessed for inclusion in the review according to the eligibility criteria. If the existing review included records of any ongoing studies that are now complete, or studies for which classification as included or excluded was pending, newly available information should be sought and, where possible, final inclusion decisions made.

If new studies are to be included in the updated review, data should be collected (see Chapter 5 ) and risk of bias assessments completed for all new studies (see Chapter 7 ). On a practical note, when changes have been made to the scope or PICO of the review, tools such as the original data collection forms may need to be altered or extended and piloted again to ensure they are fit for purpose. This may also be needed if new software tools are to be used for data collection, or if a new author team has taken on the review, although existing templates and forms may be available from the original review authors or repositories such as the Systematic Review Data Repository ( https://srdr.ahrq.gov/ ).

The findings of any new studies should be integrated into the synthesis of the review (see Chapter 10 , Chapter 11 , and Chapter 12 ), and GRADE assessments completed (or revised), taking full account of the new body of evidence (see Chapter 14 ).

If no new studies are found to be included in the review, authors should complete and publish the updated review (see Section IV.5 ). While not modifying the findings, including the details of an updated search will reassure readers and decision makers of the currency of the review.

See the online MECIR Manual for expectations relevant to conducting an update.

IV.4.1 Updating data from previously included studies

Since the time of publication, additional information may be available about one or more studies included in the existing review. For example, additional outcome data measured at later time points may now be available, or the study may have been corrected or retracted due to errors, fraud or a range of other reasons. It is important to search online journals or databases such as MEDLINE (if the study is indexed there) for any notifications, corrections or retractions.

Any additions or corrections should be incorporated into the information contained in the review, if relevant. The reasons for retraction of any included studies should be considered. In addition to the publication record, this information may be available in reports of investigations, such as by the authors’ institutions or funders. In those cases where data appear to be incorrect or possibly fabricated, they should be removed from the review analysis and this decision should be reported in the review. Other studies by the same author(s) which would also be eligible for inclusion should be checked for similar issues, and a decision made as to whether they should similarly be removed. Further guidance on identifying corrected or retracted studies is provided in Chapter 4, section 4.4.6 , and in the Cochrane policy for managing potentially problematic studies .

If a new comparison or a new outcome has been added to the review, it may be necessary to go back to the original included studies and check whether they included any information not previously collected that would be relevant to the update.

IV.5 Reporting an updated review

An updated review should meet the same standards of reporting as any review (see Chapter III ), while ensuring that all updated information and changes made to the scope and methods of the review are reported clearly. The details of any changes, including justifications for the decisions made, can be briefly documented at the beginning of the Methods section of the review and elaborated on additional supplementary material if they are significant. Authors should clearly alert readers that this is an update of an earlier version, including statements in the Abstract, Background and Protocol and registration sections of the review.

Appearing at the beginning of the review, the Background section is not directly impacted by an update, but authors may wish to review the content of the Background to ensure that it remains fit for purpose. Discussions of the prevalence or incidence of a condition, new insights into the mechanism of action or impact on populations, or descriptions of current practice or policy options may be updated. Up-to-date references should be supplied to support this information. Any references to time, such as words like ‘recently’ or ‘in the next five years’, should be amended or, if possible, removed.

Reporting the details of the updated search alongside the search information in the existing review can become quite complex, especially if there have been several updates to the review over time. Detailed information on search strategies will be reported in Cochrane Reviews as supplementary material, so does not need to be described at length in the text of the review. There are several approaches to reporting the results of an updated search:

1. An integrated approach describes all searches together, which may be most feasible if the same search was repeated.

2. An incremental approach adds information at each update to describe explicitly which searches were done for the update, retaining all information about previous searches.

3. A replacement approach describes only the searches done for the update, using the previous review as one source of studies.

If any of the sources originally searched were not searched for the update, this should be explained and justified.

The updated search should also be presented in a PRISMA-type flow diagram (see Chapter 4, Section 4.5 ). Again, there are options as to how to present the results of multiple searches coherently in the diagram. Authors can retain the results of previous searches in the review and supplement with information about studies identified in the update or, alternatively, present only information about searches in the current update, with the previous version of the review serving as one particular source of studies. If taking the latter approach, the flow diagram should show one box for the number of studies included in the original review or previous update and an additional box for the new studies retrieved for the current update. If multiple searches have been conducted for the current update, the results of all the searches should be added together. It may be helpful to consider the clarity of the diagram as a summary for readers when selecting an approach.

The methods and results described throughout the review and its summaries (including the ‘Summary of findings’ table, Abstract and Plain Language Summaries) should be checked to ensure they still reflect the methods used accurately. Where the review is considered a ‘living’ systematic review, and regular updates are planned, additional methods should be included to describe the timing and nature of this process (see Chapter 22, Section 22.2 ).

The extent of revision to the Results of the review will depend on the influence of the new data on the results of the review. Examples include:

  • the addition of small studies bringing about no change in the results or conclusions of the review (and so requiring very little revision of the text);
  • increased certainty of pre-existing results and conclusions (requiring some modification of the text); and
  • a change in the conclusion of a review (requiring a major rewrite of the Results, Discussion, Conclusion, ‘Summary of findings’ table, Abstract and Plain Language Summary).

When reporting the results, it is more helpful to readers to present an integrated picture of the overall results, rather than sequential or separate results for the update (especially where there has been more than one update), although any particularly notable changes to the review’s conclusions may be of interest to discuss when interpreting the results.

Authors should check that nothing else in the review requires editing, such as references to other Cochrane Reviews that may have been updated, or additions to the Acknowledgements. The ‘Declarations of interest’ sections of the review should be updated.

Finally, to inform returning readers, authors should summarize key changes in the ‘What’s new’ section. This should include the number of new studies and participants in those studies, and the nature of any changes in findings, the certainty of the evidence (e.g. using GRADE) and in the implications for practice.

IV.5.1 Changes in authorship

If there is a change in the authorship of the review, such as new authors joining the team, or an entirely new team of authors updating the review, the by-line (list of authors) may need to be changed. The decision regarding who is named in the by-line of an updated review, and in what order, should be assessed in terms of contributions to content in the updated version of the review (which will include historical content), and responsibility for approving the final content of the manuscript. If an author is no longer actively contributing to or involved in the approval of an updated review, the author should not be listed in the by-line of the new version and should be named in the Acknowledgements section. In addition, the contributions of all authors to both the update and earlier versions of the review should be described in the ‘Contributions of authors’ section.

See Cochrane’s policy on authorship and contributorship for Cochrane Reviews for more information.

IV.6 Chapter information

Authors: Miranda Cumpston and Ella Flemyng

Acknowledgements : This chapter builds on earlier versions of the Handbook . Contributors to earlier versions include Jacqueline Chandler, Julian Higgins, Rachel Marshall, Ruth Foxlee and members of the former Updating Working Group (Mike Clarke, Mark Davies, Davina Ghersi, Sally Green, Sonja Henderson, Harriet MacLehose, Jessie McGowan, David Moher, Rob Scholten (convenor) and Phil Wiffen). David Tovey, Carol Lefebvre and Sally Hopewell provided comments on earlier versions. Rachel Marshall re-drafted version 5.1 on which this version was based with input from Harriet MacLehose. Mona Nasser contributed to section IV.2.1. Rachel Churchill contributed to the re-structuring of this version. The work of Garner and colleagues (Garner et al 2016), a key reference used throughout, was based on a consensus meeting of experts funded by Cochrane.

IV.7 References

Adams SP, Tsang M, Wright JM. Lipid lowering efficacy of atorvastatin. Cochrane Database of Systematic Reviews 2012; 12 : CD008226.

Ahmadzai N, Newberry SJ, Maglione MA, Tsertsvadze A, Ansari MT, Hempel S, Motala A, Tsouros S, Schneider Chafen JJ, Shanman R, Moher D, Shekelle PG. A surveillance system to assess the need for updating systematic reviews. Systematic Reviews 2013; 2 : 104.

Elliott JH, Synnot A, Turner T, Simmonds M, Akl EA, McDonald S, Salanti G, Meerpohl J, MacLehose H, Hilton J, Tovey D, Shemilt I, Thomas J, Living Systematic Review N. Living systematic review: 1. Introduction-the why, what, when, and how. Journal of Clinical Epidemiology 2017; 91 : 23-30.

Garner P, Hopewell S, Chandler J, MacLehose H, Schünemann HJ, Akl EA, Beyene J, Chang S, Churchill R, Dearness K, Guyatt G, Lefebvre C, Liles B, Marshall R, Martinez Garcia L, Mavergames C, Nasser M, Qaseem A, Sampson M, Soares-Weiser K, Takwoingi Y, Thabane L, Trivella M, Tugwell P, Welsh E, Wilson EC, Schünemann HJ, Panel for Updating Guidance for Systematic Reviews (PUGs). When and how to update systematic reviews: consensus and checklist. BMJ 2016; 354 : i3507.

Green S, Buchbinder R, Hetrick S. Physiotherapy interventions for shoulder pain. Cochrane Database of Systematic Reviews 2003; 2 : CD004258.

Higgins JPT. Convincing evidence from controlled and uncontrolled studies on the lipid-lowering effect of a statin. Cochrane Database of Systematic Reviews 2012: ED000049.

Page MJ, Green S, Kramer S, Johnston RV, McBain B, Buchbinder R. Electrotherapy modalities for adhesive capsulitis (frozen shoulder). Cochrane Database of Systematic Reviews 2014a; 10 : CD011324.

Page MJ, Green S, Kramer S, Johnston RV, McBain B, Chau M, Buchbinder R. Manual therapy and exercise for adhesive capsulitis (frozen shoulder). Cochrane Database of Systematic Reviews 2014b; 8 : CD011275.

Page MJ, Green S, McBain B, Surace SJ, Deitch J, Lyttle N, Mrocki MA, Buchbinder R. Manual therapy and exercise for rotator cuff disease. Cochrane Database of Systematic Reviews 2016a; 6 : CD012224.

Page MJ, Green S, Mrocki MA, Surace SJ, Deitch J, McBain B, Lyttle N, Buchbinder R. Electrotherapy modalities for rotator cuff disease. Cochrane Database of Systematic Reviews 2016b; 6 : CD012225.

Prasad K, Singh MB, Ryan H. Corticosteroids for managing tuberculous meningitis. Cochrane Database of Systematic Reviews 2016; 4 : CD002244.

Sampson M, Shojania KG, McGowan J, Daniel R, Rader T, Iansavichene AE, Ji J, Ansari MT, Moher D. Surveillance search techniques identified the need to update systematic reviews. Journal of Clinical Epidemiology 2008; 61 : 755-762.

Shea B, Boers M, Grimshaw JM, Hamel C, Bouter LM. Does updating improve the methodological and reporting quality of systematic reviews? BMC Medical Research Methodology 2006; 6 : 27.

Shekelle P, Newberry S, Wu H, Suttorp M, Motala A, Lim Y, et al. Identifying Signals for Updating Systematic Reviews: A Comparison of Two Methods (Prepared by: The RAND Corporation, Southern California Evidence-based Practice Center, Santa Monica, CA under Contract No 290-2007-10062I; Tufts Evidence-based Practice Center, Tufts Medical Center, Boston, MA under Contract No 290-2007-10055I; University of Ottawa Evidence-based Practice Center, Ottawa, Canada under Contract No 290-2007-10059I). Rockville (MD): Agency for Healthcare Research and Quality; 2011.

Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? A survival analysis. Annals of Internal Medicine 2007; 147 : 224-233.

Takwoingi Y, Hopewell S, Tovey D, Sutton AJ. A multicomponent decision tool for prioritising the updating of systematic reviews. BMJ 2013; 347 : f7191.

Tovey D, Marshall R, Bazian Ltd, Hopewell S, Rader T. National Institute for Health Research Cochrane-National Health Service Engagement Award Scheme Fit for purpose: centralised updating support for high-priority Cochrane reviews 2011. http://www.editorial-unit.cochrane.org/fit-purpose-centralised-updating-support-high-priority-cochrane-reviews .

Tsertsvadze A, Maglione M, Chou R, Garritty C, Coleman C, Lux L, Bass E, Balshem H, Moher D. Updating comparative effectiveness reviews: Current efforts in AHRQ's Effective Health Care Program. Journal of Clinical Epidemiology 2011; 64 : 1208-1215.

Zani B, Gathu M, Donegan S, Olliaro PL, Sinclair D. Dihydroartemisinin-piperaquine for treating uncomplicated Plasmodium falciparum malaria. Cochrane Database of Systematic Reviews 2014; 1 : CD010927.

For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.

University of Texas

  • University of Texas Libraries

Literature Reviews

  • What is a literature review?
  • Steps in the Literature Review Process
  • Define your research question
  • Determine inclusion and exclusion criteria
  • Choose databases and search
  • Review Results
  • Synthesize Results
  • Analyze Results
  • Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

  • Provide a comprehensive and updated review of the literature;
  • Explain why this review has taken place;
  • Articulate a position or hypothesis;
  • Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

  • Demonstrate how a study fills a gap in research
  • Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

  • Organizes or describes a topic
  • Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

  • It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
  • It may be too focused. Some niche studies may miss the bigger picture.
  • It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
  • It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy ​

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

  • October 26, 2022 recording
  • Last Updated: Oct 26, 2022 2:49 PM
  • URL: https://guides.lib.utexas.edu/literaturereviews

Creative Commons License

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base

Methodology

  • How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

  • Search for relevant literature
  • Evaluate sources
  • Identify themes, debates, and gaps
  • Outline the structure
  • Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

upload-your-document-ai-proofreader

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

  • Quick Run-through
  • Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

  • Demonstrate your familiarity with the topic and its scholarly context
  • Develop a theoretical framework and methodology for your research
  • Position your work in relation to other researchers and theorists
  • Show how your research addresses a gap or contributes to a debate
  • Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Literature review guide

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

  • Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
  • Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
  • Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
  • Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

  • Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
  • Body image, self-perception, self-esteem, mental health
  • Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

  • Your university’s library catalogue
  • Google Scholar
  • Project Muse (humanities and social sciences)
  • Medline (life sciences and biomedicine)
  • EconLit (economics)
  • Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

  • What question or problem is the author addressing?
  • What are the key concepts and how are they defined?
  • What are the key theories, models, and methods?
  • Does the research use established frameworks or take an innovative approach?
  • What are the results and conclusions of the study?
  • How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
  • What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

an updated review of the literature

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

  • Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
  • Themes: what questions or concepts recur across the literature?
  • Debates, conflicts and contradictions: where do sources disagree?
  • Pivotal publications: are there any influential theories or studies that changed the direction of the field?
  • Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

  • Most research has focused on young women.
  • There is an increasing interest in the visual aspects of social media.
  • But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

  • Look at what results have emerged in qualitative versus quantitative research
  • Discuss how the topic has been approached by empirical versus theoretical scholarship
  • Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

  • Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically evaluate: mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

  • Sampling methods
  • Simple random sampling
  • Stratified sampling
  • Cluster sampling
  • Likert scales
  • Reproducibility

 Statistics

  • Null hypothesis
  • Statistical power
  • Probability distribution
  • Effect size
  • Poisson distribution

Research bias

  • Optimism bias
  • Cognitive bias
  • Implicit bias
  • Hawthorne effect
  • Anchoring bias
  • Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

  • To familiarize yourself with the current state of knowledge on your topic
  • To ensure that you’re not just repeating what others have already done
  • To identify gaps in knowledge and unresolved problems that your research can address
  • To develop your theoretical framework and methodology
  • To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved February 22, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, what is your plagiarism score.

  • Systematic review
  • Open access
  • Published: 19 February 2024

‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice

  • Annette Boaz   ORCID: orcid.org/0000-0003-0557-1294 1 ,
  • Juan Baeza 2 ,
  • Alec Fraser   ORCID: orcid.org/0000-0003-1121-1551 2 &
  • Erik Persson 3  

Implementation Science volume  19 , Article number:  15 ( 2024 ) Cite this article

1758 Accesses

68 Altmetric

Metrics details

The gap between research findings and clinical practice is well documented and a range of strategies have been developed to support the implementation of research into clinical practice. The objective of this study was to update and extend two previous reviews of systematic reviews of strategies designed to implement research evidence into clinical practice.

We developed a comprehensive systematic literature search strategy based on the terms used in the previous reviews to identify studies that looked explicitly at interventions designed to turn research evidence into practice. The search was performed in June 2022 in four electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched from January 2010 up to June 2022 and applied no language restrictions. Two independent reviewers appraised the quality of included studies using a quality assessment checklist. To reduce the risk of bias, papers were excluded following discussion between all members of the team. Data were synthesised using descriptive and narrative techniques to identify themes and patterns linked to intervention strategies, targeted behaviours, study settings and study outcomes.

We identified 32 reviews conducted between 2010 and 2022. The reviews are mainly of multi-faceted interventions ( n  = 20) although there are reviews focusing on single strategies (ICT, educational, reminders, local opinion leaders, audit and feedback, social media and toolkits). The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Furthermore, a lot of nuance lies behind these headline findings, and this is increasingly commented upon in the reviews themselves.

Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been identified. We need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of research perspectives (including social science) in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed.

Peer Review reports

Contribution to the literature

Considerable time and money is invested in implementing and evaluating strategies to increase the implementation of research into clinical practice.

The growing body of evidence is not providing the anticipated clear lessons to support improved implementation.

Instead what is needed is better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice.

This would involve a more central role in implementation science for a wider range of perspectives, especially from the social, economic, political and behavioural sciences and for greater use of different types of synthesis, such as realist synthesis.

Introduction

The gap between research findings and clinical practice is well documented and a range of interventions has been developed to increase the implementation of research into clinical practice [ 1 , 2 ]. In recent years researchers have worked to improve the consistency in the ways in which these interventions (often called strategies) are described to support their evaluation. One notable development has been the emergence of Implementation Science as a field focusing explicitly on “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice” ([ 3 ] p. 1). The work of implementation science focuses on closing, or at least narrowing, the gap between research and practice. One contribution has been to map existing interventions, identifying 73 discreet strategies to support research implementation [ 4 ] which have been grouped into 9 clusters [ 5 ]. The authors note that they have not considered the evidence of effectiveness of the individual strategies and that a next step is to understand better which strategies perform best in which combinations and for what purposes [ 4 ]. Other authors have noted that there is also scope to learn more from other related fields of study such as policy implementation [ 6 ] and to draw on methods designed to support the evaluation of complex interventions [ 7 ].

The increase in activity designed to support the implementation of research into practice and improvements in reporting provided the impetus for an update of a review of systematic reviews of the effectiveness of interventions designed to support the use of research in clinical practice [ 8 ] which was itself an update of the review conducted by Grimshaw and colleagues in 2001. The 2001 review [ 9 ] identified 41 reviews considering a range of strategies including educational interventions, audit and feedback, computerised decision support to financial incentives and combined interventions. The authors concluded that all the interventions had the potential to promote the uptake of evidence in practice, although no one intervention seemed to be more effective than the others in all settings. They concluded that combined interventions were more likely to be effective than single interventions. The 2011 review identified a further 13 systematic reviews containing 313 discrete primary studies. Consistent with the previous review, four main strategy types were identified: audit and feedback; computerised decision support; opinion leaders; and multi-faceted interventions (MFIs). Nine of the reviews reported on MFIs. The review highlighted the small effects of single interventions such as audit and feedback, computerised decision support and opinion leaders. MFIs claimed an improvement in effectiveness over single interventions, although effect sizes remained small to moderate and this improvement in effectiveness relating to MFIs has been questioned in a subsequent review [ 10 ]. In updating the review, we anticipated a larger pool of reviews and an opportunity to consolidate learning from more recent systematic reviews of interventions.

This review updates and extends our previous review of systematic reviews of interventions designed to implement research evidence into clinical practice. To identify potentially relevant peer-reviewed research papers, we developed a comprehensive systematic literature search strategy based on the terms used in the Grimshaw et al. [ 9 ] and Boaz, Baeza and Fraser [ 8 ] overview articles. To ensure optimal retrieval, our search strategy was refined with support from an expert university librarian, considering the ongoing improvements in the development of search filters for systematic reviews since our first review [ 11 ]. We also wanted to include technology-related terms (e.g. apps, algorithms, machine learning, artificial intelligence) to find studies that explored interventions based on the use of technological innovations as mechanistic tools for increasing the use of evidence into practice (see Additional file 1 : Appendix A for full search strategy).

The search was performed in June 2022 in the following electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched for articles published since the 2011 review. We searched from January 2010 up to June 2022 and applied no language restrictions. Reference lists of relevant papers were also examined.

We uploaded the results using EPPI-Reviewer, a web-based tool that facilitated semi-automation of the screening process and removal of duplicate studies. We made particular use of a priority screening function to reduce screening workload and avoid ‘data deluge’ [ 12 ]. Through machine learning, one reviewer screened a smaller number of records ( n  = 1200) to train the software to predict whether a given record was more likely to be relevant or irrelevant, thus pulling the relevant studies towards the beginning of the screening process. This automation did not replace manual work but helped the reviewer to identify eligible studies more quickly. During the selection process, we included studies that looked explicitly at interventions designed to turn research evidence into practice. Studies were included if they met the following pre-determined inclusion criteria:

The study was a systematic review

Search terms were included

Focused on the implementation of research evidence into practice

The methodological quality of the included studies was assessed as part of the review

Study populations included healthcare providers and patients. The EPOC taxonomy [ 13 ] was used to categorise the strategies. The EPOC taxonomy has four domains: delivery arrangements, financial arrangements, governance arrangements and implementation strategies. The implementation strategies domain includes 20 strategies targeted at healthcare workers. Numerous EPOC strategies were assessed in the review including educational strategies, local opinion leaders, reminders, ICT-focused approaches and audit and feedback. Some strategies that did not fit easily within the EPOC categories were also included. These were social media strategies and toolkits, and multi-faceted interventions (MFIs) (see Table  2 ). Some systematic reviews included comparisons of different interventions while other reviews compared one type of intervention against a control group. Outcomes related to improvements in health care processes or patient well-being. Numerous individual study types (RCT, CCT, BA, ITS) were included within the systematic reviews.

We excluded papers that:

Focused on changing patient rather than provider behaviour

Had no demonstrable outcomes

Made unclear or no reference to research evidence

The last of these criteria was sometimes difficult to judge, and there was considerable discussion amongst the research team as to whether the link between research evidence and practice was sufficiently explicit in the interventions analysed. As we discussed in the previous review [ 8 ] in the field of healthcare, the principle of evidence-based practice is widely acknowledged and tools to change behaviour such as guidelines are often seen to be an implicit codification of evidence, despite the fact that this is not always the case.

Reviewers employed a two-stage process to select papers for inclusion. First, all titles and abstracts were screened by one reviewer to determine whether the study met the inclusion criteria. Two papers [ 14 , 15 ] were identified that fell just before the 2010 cut-off. As they were not identified in the searches for the first review [ 8 ] they were included and progressed to assessment. Each paper was rated as include, exclude or maybe. The full texts of 111 relevant papers were assessed independently by at least two authors. To reduce the risk of bias, papers were excluded following discussion between all members of the team. 32 papers met the inclusion criteria and proceeded to data extraction. The study selection procedure is documented in a PRISMA literature flow diagram (see Fig.  1 ). We were able to include French, Spanish and Portuguese papers in the selection reflecting the language skills in the study team, but none of the papers identified met the inclusion criteria. Other non- English language papers were excluded.

figure 1

PRISMA flow diagram. Source: authors

One reviewer extracted data on strategy type, number of included studies, local, target population, effectiveness and scope of impact from the included studies. Two reviewers then independently read each paper and noted key findings and broad themes of interest which were then discussed amongst the wider authorial team. Two independent reviewers appraised the quality of included studies using a Quality Assessment Checklist based on Oxman and Guyatt [ 16 ] and Francke et al. [ 17 ]. Each study was rated a quality score ranging from 1 (extensive flaws) to 7 (minimal flaws) (see Additional file 2 : Appendix B). All disagreements were resolved through discussion. Studies were not excluded in this updated overview based on methodological quality as we aimed to reflect the full extent of current research into this topic.

The extracted data were synthesised using descriptive and narrative techniques to identify themes and patterns in the data linked to intervention strategies, targeted behaviours, study settings and study outcomes.

Thirty-two studies were included in the systematic review. Table 1. provides a detailed overview of the included systematic reviews comprising reference, strategy type, quality score, number of included studies, local, target population, effectiveness and scope of impact (see Table  1. at the end of the manuscript). Overall, the quality of the studies was high. Twenty-three studies scored 7, six studies scored 6, one study scored 5, one study scored 4 and one study scored 3. The primary focus of the review was on reviews of effectiveness studies, but a small number of reviews did include data from a wider range of methods including qualitative studies which added to the analysis in the papers [ 18 , 19 , 20 , 21 ]. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. In this section, we discuss the different EPOC-defined implementation strategies in turn. Interestingly, we found only two ‘new’ approaches in this review that did not fit into the existing EPOC approaches. These are a review focused on the use of social media and a review considering toolkits. In addition to single interventions, we also discuss multi-faceted interventions. These were the most common intervention approach overall. A summary is provided in Table  2 .

Educational strategies

The overview identified three systematic reviews focusing on educational strategies. Grudniewicz et al. [ 22 ] explored the effectiveness of printed educational materials on primary care physician knowledge, behaviour and patient outcomes and concluded they were not effective in any of these aspects. Koota, Kääriäinen and Melender [ 23 ] focused on educational interventions promoting evidence-based practice among emergency room/accident and emergency nurses and found that interventions involving face-to-face contact led to significant or highly significant effects on patient benefits and emergency nurses’ knowledge, skills and behaviour. Interventions using written self-directed learning materials also led to significant improvements in nurses’ knowledge of evidence-based practice. Although the quality of the studies was high, the review primarily included small studies with low response rates, and many of them relied on self-assessed outcomes; consequently, the strength of the evidence for these outcomes is modest. Wu et al. [ 20 ] questioned if educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes. Although based on evaluation projects and qualitative data, their results also suggest that positive changes on patient outcomes can be made following the implementation of specific evidence-based approaches (or projects). The differing positive outcomes for educational strategies aimed at nurses might indicate that the target audience is important.

Local opinion leaders

Flodgren et al. [ 24 ] was the only systemic review focusing solely on opinion leaders. The review found that local opinion leaders alone, or in combination with other interventions, can be effective in promoting evidence‐based practice, but this varies both within and between studies and the effect on patient outcomes is uncertain. The review found that, overall, any intervention involving opinion leaders probably improves healthcare professionals’ compliance with evidence-based practice but varies within and across studies. However, how opinion leaders had an impact could not be determined because of insufficient details were provided, illustrating that reporting specific details in published studies is important if diffusion of effective methods of increasing evidence-based practice is to be spread across a system. The usefulness of this review is questionable because it cannot provide evidence of what is an effective opinion leader, whether teams of opinion leaders or a single opinion leader are most effective, or the most effective methods used by opinion leaders.

Pantoja et al. [ 26 ] was the only systemic review focusing solely on manually generated reminders delivered on paper included in the overview. The review explored how these affected professional practice and patient outcomes. The review concluded that manually generated reminders delivered on paper as a single intervention probably led to small to moderate increases in adherence to clinical recommendations, and they could be used as a single quality improvement intervention. However, the authors indicated that this intervention would make little or no difference to patient outcomes. The authors state that such a low-tech intervention may be useful in low- and middle-income countries where paper records are more likely to be the norm.

ICT-focused approaches

The three ICT-focused reviews [ 14 , 27 , 28 ] showed mixed results. Jamal, McKenzie and Clark [ 14 ] explored the impact of health information technology on the quality of medical and health care. They examined the impact of electronic health record, computerised provider order-entry, or decision support system. This showed a positive improvement in adherence to evidence-based guidelines but not to patient outcomes. The number of studies included in the review was low and so a conclusive recommendation could not be reached based on this review. Similarly, Brown et al. [ 28 ] found that technology-enabled knowledge translation interventions may improve knowledge of health professionals, but all eight studies raised concerns of bias. The De Angelis et al. [ 27 ] review was more promising, reporting that ICT can be a good way of disseminating clinical practice guidelines but conclude that it is unclear which type of ICT method is the most effective.

Audit and feedback

Sykes, McAnuff and Kolehmainen [ 29 ] examined whether audit and feedback were effective in dementia care and concluded that it remains unclear which ingredients of audit and feedback are successful as the reviewed papers illustrated large variations in the effectiveness of interventions using audit and feedback.

Non-EPOC listed strategies: social media, toolkits

There were two new (non-EPOC listed) intervention types identified in this review compared to the 2011 review — fewer than anticipated. We categorised a third — ‘care bundles’ [ 36 ] as a multi-faceted intervention due to its description in practice and a fourth — ‘Technology Enhanced Knowledge Transfer’ [ 28 ] was classified as an ICT-focused approach. The first new strategy was identified in Bhatt et al.’s [ 30 ] systematic review of the use of social media for the dissemination of clinical practice guidelines. They reported that the use of social media resulted in a significant improvement in knowledge and compliance with evidence-based guidelines compared with more traditional methods. They noted that a wide selection of different healthcare professionals and patients engaged with this type of social media and its global reach may be significant for low- and middle-income countries. This review was also noteworthy for developing a simple stepwise method for using social media for the dissemination of clinical practice guidelines. However, it is debatable whether social media can be classified as an intervention or just a different way of delivering an intervention. For example, the review discussed involving opinion leaders and patient advocates through social media. However, this was a small review that included only five studies, so further research in this new area is needed. Yamada et al. [ 31 ] draw on 39 studies to explore the application of toolkits, 18 of which had toolkits embedded within larger KT interventions, and 21 of which evaluated toolkits as standalone interventions. The individual component strategies of the toolkits were highly variable though the authors suggest that they align most closely with educational strategies. The authors conclude that toolkits as either standalone strategies or as part of MFIs hold some promise for facilitating evidence use in practice but caution that the quality of many of the primary studies included is considered weak limiting these findings.

Multi-faceted interventions

The majority of the systematic reviews ( n  = 20) reported on more than one intervention type. Some of these systematic reviews focus exclusively on multi-faceted interventions, whilst others compare different single or combined interventions aimed at achieving similar outcomes in particular settings. While these two approaches are often described in a similar way, they are actually quite distinct from each other as the former report how multiple strategies may be strategically combined in pursuance of an agreed goal, whilst the latter report how different strategies may be incidentally used in sometimes contrasting settings in the pursuance of similar goals. Ariyo et al. [ 35 ] helpfully summarise five key elements often found in effective MFI strategies in LMICs — but which may also be transferrable to HICs. First, effective MFIs encourage a multi-disciplinary approach acknowledging the roles played by different professional groups to collectively incorporate evidence-informed practice. Second, they utilise leadership drawing on a wide set of clinical and non-clinical actors including managers and even government officials. Third, multiple types of educational practices are utilised — including input from patients as stakeholders in some cases. Fourth, protocols, checklists and bundles are used — most effectively when local ownership is encouraged. Finally, most MFIs included an emphasis on monitoring and evaluation [ 35 ]. In contrast, other studies offer little information about the nature of the different MFI components of included studies which makes it difficult to extrapolate much learning from them in relation to why or how MFIs might affect practice (e.g. [ 28 , 38 ]). Ultimately, context matters, which some review authors argue makes it difficult to say with real certainty whether single or MFI strategies are superior (e.g. [ 21 , 27 ]). Taking all the systematic reviews together we may conclude that MFIs appear to be more likely to generate positive results than single interventions (e.g. [ 34 , 45 ]) though other reviews should make us cautious (e.g. [ 32 , 43 ]).

While multi-faceted interventions still seem to be more effective than single-strategy interventions, there were important distinctions between how the results of reviews of MFIs are interpreted in this review as compared to the previous reviews [ 8 , 9 ], reflecting greater nuance and debate in the literature. This was particularly noticeable where the effectiveness of MFIs was compared to single strategies, reflecting developments widely discussed in previous studies [ 10 ]. We found that most systematic reviews are bounded by their clinical, professional, spatial, system, or setting criteria and often seek to draw out implications for the implementation of evidence in their areas of specific interest (such as nursing or acute care). Frequently this means combining all relevant studies to explore the respective foci of each systematic review. Therefore, most reviews we categorised as MFIs actually include highly variable numbers and combinations of intervention strategies and highly heterogeneous original study designs. This makes statistical analyses of the type used by Squires et al. [ 10 ] on the three reviews in their paper not possible. Further, it also makes extrapolating findings and commenting on broad themes complex and difficult. This may suggest that future research should shift its focus from merely examining ‘what works’ to ‘what works where and what works for whom’ — perhaps pointing to the value of realist approaches to these complex review topics [ 48 , 49 ] and other more theory-informed approaches [ 50 ].

Some reviews have a relatively small number of studies (i.e. fewer than 10) and the authors are often understandably reluctant to engage with wider debates about the implications of their findings. Other larger studies do engage in deeper discussions about internal comparisons of findings across included studies and also contextualise these in wider debates. Some of the most informative studies (e.g. [ 35 , 40 ]) move beyond EPOC categories and contextualise MFIs within wider systems thinking and implementation theory. This distinction between MFIs and single interventions can actually be very useful as it offers lessons about the contexts in which individual interventions might have bounded effectiveness (i.e. educational interventions for individual change). Taken as a whole, this may also then help in terms of how and when to conjoin single interventions into effective MFIs.

In the two previous reviews, a consistent finding was that MFIs were more effective than single interventions [ 8 , 9 ]. However, like Squires et al. [ 10 ] this overview is more equivocal on this important issue. There are four points which may help account for the differences in findings in this regard. Firstly, the diversity of the systematic reviews in terms of clinical topic or setting is an important factor. Secondly, there is heterogeneity of the studies within the included systematic reviews themselves. Thirdly, there is a lack of consistency with regards to the definition and strategies included within of MFIs. Finally, there are epistemological differences across the papers and the reviews. This means that the results that are presented depend on the methods used to measure, report, and synthesise them. For instance, some reviews highlight that education strategies can be useful to improve provider understanding — but without wider organisational or system-level change, they may struggle to deliver sustained transformation [ 19 , 44 ].

It is also worth highlighting the importance of the theory of change underlying the different interventions. Where authors of the systematic reviews draw on theory, there is space to discuss/explain findings. We note a distinction between theoretical and atheoretical systematic review discussion sections. Atheoretical reviews tend to present acontextual findings (for instance, one study found very positive results for one intervention, and this gets highlighted in the abstract) whilst theoretically informed reviews attempt to contextualise and explain patterns within the included studies. Theory-informed systematic reviews seem more likely to offer more profound and useful insights (see [ 19 , 35 , 40 , 43 , 45 ]). We find that the most insightful systematic reviews of MFIs engage in theoretical generalisation — they attempt to go beyond the data of individual studies and discuss the wider implications of the findings of the studies within their reviews drawing on implementation theory. At the same time, they highlight the active role of context and the wider relational and system-wide issues linked to implementation. It is these types of investigations that can help providers further develop evidence-based practice.

This overview has identified a small, but insightful set of papers that interrogate and help theorise why, how, for whom, and in which circumstances it might be the case that MFIs are superior (see [ 19 , 35 , 40 ] once more). At the level of this overview — and in most of the systematic reviews included — it appears to be the case that MFIs struggle with the question of attribution. In addition, there are other important elements that are often unmeasured, or unreported (e.g. costs of the intervention — see [ 40 ]). Finally, the stronger systematic reviews [ 19 , 35 , 40 , 43 , 45 ] engage with systems issues, human agency and context [ 18 ] in a way that was not evident in the systematic reviews identified in the previous reviews [ 8 , 9 ]. The earlier reviews lacked any theory of change that might explain why MFIs might be more effective than single ones — whereas now some systematic reviews do this, which enables them to conclude that sometimes single interventions can still be more effective.

As Nilsen et al. ([ 6 ] p. 7) note ‘Study findings concerning the effectiveness of various approaches are continuously synthesized and assembled in systematic reviews’. We may have gone as far as we can in understanding the implementation of evidence through systematic reviews of single and multi-faceted interventions and the next step would be to conduct more research exploring the complex and situated nature of evidence used in clinical practice and by particular professional groups. This would further build on the nuanced discussion and conclusion sections in a subset of the papers we reviewed. This might also support the field to move away from isolating individual implementation strategies [ 6 ] to explore the complex processes involving a range of actors with differing capacities [ 51 ] working in diverse organisational cultures. Taxonomies of implementation strategies do not fully account for the complex process of implementation, which involves a range of different actors with different capacities and skills across multiple system levels. There is plenty of work to build on, particularly in the social sciences, which currently sits at the margins of debates about evidence implementation (see for example, Normalisation Process Theory [ 52 ]).

There are several changes that we have identified in this overview of systematic reviews in comparison to the review we published in 2011 [ 8 ]. A consistent and welcome finding is that the overall quality of the systematic reviews themselves appears to have improved between the two reviews, although this is not reflected upon in the papers. This is exhibited through better, clearer reporting mechanisms in relation to the mechanics of the reviews, alongside a greater attention to, and deeper description of, how potential biases in included papers are discussed. Additionally, there is an increased, but still limited, inclusion of original studies conducted in low- and middle-income countries as opposed to just high-income countries. Importantly, we found that many of these systematic reviews are attuned to, and comment upon the contextual distinctions of pursuing evidence-informed interventions in health care settings in different economic settings. Furthermore, systematic reviews included in this updated article cover a wider set of clinical specialities (both within and beyond hospital settings) and have a focus on a wider set of healthcare professions — discussing both similarities, differences and inter-professional challenges faced therein, compared to the earlier reviews. These wider ranges of studies highlight that a particular intervention or group of interventions may work well for one professional group but be ineffective for another. This diversity of study settings allows us to consider the important role context (in its many forms) plays on implementing evidence into practice. Examining the complex and varied context of health care will help us address what Nilsen et al. ([ 6 ] p. 1) described as, ‘society’s health problems [that] require research-based knowledge acted on by healthcare practitioners together with implementation of political measures from governmental agencies’. This will help us shift implementation science to move, ‘beyond a success or failure perspective towards improved analysis of variables that could explain the impact of the implementation process’ ([ 6 ] p. 2).

This review brings together 32 papers considering individual and multi-faceted interventions designed to support the use of evidence in clinical practice. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been conducted. As a whole, this substantial body of knowledge struggles to tell us more about the use of individual and MFIs than: ‘it depends’. To really move forwards in addressing the gap between research evidence and practice, we may need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of perspectives, especially from the social, economic, political and behavioural sciences in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed. Harvey et al. [ 53 ] suggest that when context is likely to be critical to implementation success there are a range of primary research approaches (participatory research, realist evaluation, developmental evaluation, ethnography, quality/ rapid cycle improvement) that are likely to be appropriate and insightful. While these approaches often form part of implementation studies in the form of process evaluations, they are usually relatively small scale in relation to implementation research as a whole. As a result, the findings often do not make it into the subsequent systematic reviews. This review provides further evidence that we need to bring qualitative approaches in from the periphery to play a central role in many implementation studies and subsequent evidence syntheses. It would be helpful for systematic reviews, at the very least, to include more detail about the interventions and their implementation in terms of how and why they worked.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Before and after study

Controlled clinical trial

Effective Practice and Organisation of Care

High-income countries

Information and Communications Technology

Interrupted time series

Knowledge translation

Low- and middle-income countries

Randomised controlled trial

Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–30. https://doi.org/10.1016/S0140-6736(03)14546-1 .

Article   PubMed   Google Scholar  

Green LA, Seifert CM. Translation of research into practice: why we can’t “just do it.” J Am Board Fam Pract. 2005;18:541–5. https://doi.org/10.3122/jabfm.18.6.541 .

Eccles MP, Mittman BS. Welcome to Implementation Science. Implement Sci. 2006;1:1–3. https://doi.org/10.1186/1748-5908-1-1 .

Article   PubMed Central   Google Scholar  

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:2–14. https://doi.org/10.1186/s13012-015-0209-1 .

Article   Google Scholar  

Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:1–8. https://doi.org/10.1186/s13012-015-0295-0 .

Nilsen P, Ståhl C, Roback K, et al. Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Sci. 2013;8:2–12. https://doi.org/10.1186/1748-5908-8-63 .

Rycroft-Malone J, Seers K, Eldh AC, et al. A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: an exemplar. Implementation Sci. 2018;13:1–15. https://doi.org/10.1186/s13012-018-0811-0 .

Boaz A, Baeza J, Fraser A, European Implementation Score Collaborative Group (EIS). Effective implementation of research into practice: an overview of systematic reviews of the health literature. BMC Res Notes. 2011;4:212. https://doi.org/10.1186/1756-0500-4-212 .

Article   PubMed   PubMed Central   Google Scholar  

Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior – an overview of systematic reviews of interventions. Med Care. 2001;39 8Suppl 2:II2–45.

Google Scholar  

Squires JE, Sullivan K, Eccles MP, et al. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:1–22. https://doi.org/10.1186/s13012-014-0152-6 .

Salvador-Oliván JA, Marco-Cuenca G, Arquero-Avilés R. Development of an efficient search filter to retrieve systematic reviews from PubMed. J Med Libr Assoc. 2021;109:561–74. https://doi.org/10.5195/jmla.2021.1223 .

Thomas JM. Diffusion of innovation in systematic review methodology: why is study selection not yet assisted by automation? OA Evid Based Med. 2013;1:1–6.

Effective Practice and Organisation of Care (EPOC). The EPOC taxonomy of health systems interventions. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2016. epoc.cochrane.org/epoc-taxonomy . Accessed 9 Oct 2023.

Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: a systematic review. Health Inf Manag. 2009;38:26–37. https://doi.org/10.1177/183335830903800305 .

Menon A, Korner-Bitensky N, Kastner M, et al. Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med. 2009;41:1024–32. https://doi.org/10.2340/16501977-0451 .

Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44:1271–8. https://doi.org/10.1016/0895-4356(91)90160-b .

Article   CAS   PubMed   Google Scholar  

Francke AL, Smit MC, de Veer AJ, et al. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:1–11. https://doi.org/10.1186/1472-6947-8-38 .

Jones CA, Roop SC, Pohar SL, et al. Translating knowledge in rehabilitation: systematic review. Phys Ther. 2015;95:663–77. https://doi.org/10.2522/ptj.20130512 .

Scott D, Albrecht L, O’Leary K, Ball GDC, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:1–17. https://doi.org/10.1186/1748-5908-7-70 .

Wu Y, Brettle A, Zhou C, Ou J, et al. Do educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes? A systematic review. Nurse Educ Today. 2018;70:109–14. https://doi.org/10.1016/j.nedt.2018.08.026 .

Yost J, Ganann R, Thompson D, Aloweni F, et al. The effectiveness of knowledge translation interventions for promoting evidence-informed decision-making among nurses in tertiary care: a systematic review and meta-analysis. Implement Sci. 2015;10:1–15. https://doi.org/10.1186/s13012-015-0286-1 .

Grudniewicz A, Kealy R, Rodseth RN, Hamid J, et al. What is the effectiveness of printed educational materials on primary care physician knowledge, behaviour, and patient outcomes: a systematic review and meta-analyses. Implement Sci. 2015;10:2–12. https://doi.org/10.1186/s13012-015-0347-5 .

Koota E, Kääriäinen M, Melender HL. Educational interventions promoting evidence-based practice among emergency nurses: a systematic review. Int Emerg Nurs. 2018;41:51–8. https://doi.org/10.1016/j.ienj.2018.06.004 .

Flodgren G, O’Brien MA, Parmelli E, et al. Local opinion leaders: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD000125.pub5 .

Arditi C, Rège-Walther M, Durieux P, et al. Computer-generated reminders delivered on paper to healthcare professionals: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2017. https://doi.org/10.1002/14651858.CD001175.pub4 .

Pantoja T, Grimshaw JM, Colomer N, et al. Manually-generated reminders delivered on paper: effects on professional practice and patient outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD001174.pub4 .

De Angelis G, Davies B, King J, McEwan J, et al. Information and communication technologies for the dissemination of clinical practice guidelines to health professionals: a systematic review. JMIR Med Educ. 2016;2:e16. https://doi.org/10.2196/mededu.6288 .

Brown A, Barnes C, Byaruhanga J, McLaughlin M, et al. Effectiveness of technology-enabled knowledge translation strategies in improving the use of research in public health: systematic review. J Med Internet Res. 2020;22:e17274. https://doi.org/10.2196/17274 .

Sykes MJ, McAnuff J, Kolehmainen N. When is audit and feedback effective in dementia care? A systematic review. Int J Nurs Stud. 2018;79:27–35. https://doi.org/10.1016/j.ijnurstu.2017.10.013 .

Bhatt NR, Czarniecki SW, Borgmann H, et al. A systematic review of the use of social media for dissemination of clinical practice guidelines. Eur Urol Focus. 2021;7:1195–204. https://doi.org/10.1016/j.euf.2020.10.008 .

Yamada J, Shorkey A, Barwick M, Widger K, et al. The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ Open. 2015;5:e006808. https://doi.org/10.1136/bmjopen-2014-006808 .

Afari-Asiedu S, Abdulai MA, Tostmann A, et al. Interventions to improve dispensing of antibiotics at the community level in low and middle income countries: a systematic review. J Glob Antimicrob Resist. 2022;29:259–74. https://doi.org/10.1016/j.jgar.2022.03.009 .

Boonacker CW, Hoes AW, Dikhoff MJ, Schilder AG, et al. Interventions in health care professionals to improve treatment in children with upper respiratory tract infections. Int J Pediatr Otorhinolaryngol. 2010;74:1113–21. https://doi.org/10.1016/j.ijporl.2010.07.008 .

Al Zoubi FM, Menon A, Mayo NE, et al. The effectiveness of interventions designed to increase the uptake of clinical practice guidelines and best practices among musculoskeletal professionals: a systematic review. BMC Health Serv Res. 2018;18:2–11. https://doi.org/10.1186/s12913-018-3253-0 .

Ariyo P, Zayed B, Riese V, Anton B, et al. Implementation strategies to reduce surgical site infections: a systematic review. Infect Control Hosp Epidemiol. 2019;3:287–300. https://doi.org/10.1017/ice.2018.355 .

Borgert MJ, Goossens A, Dongelmans DA. What are effective strategies for the implementation of care bundles on ICUs: a systematic review. Implement Sci. 2015;10:1–11. https://doi.org/10.1186/s13012-015-0306-1 .

Cahill LS, Carey LM, Lannin NA, et al. Implementation interventions to promote the uptake of evidence-based practices in stroke rehabilitation. Cochrane Database Syst Rev. 2020. https://doi.org/10.1002/14651858.CD012575.pub2 .

Pedersen ER, Rubenstein L, Kandrack R, Danz M, et al. Elusive search for effective provider interventions: a systematic review of provider interventions to increase adherence to evidence-based treatment for depression. Implement Sci. 2018;13:1–30. https://doi.org/10.1186/s13012-018-0788-8 .

Jenkins HJ, Hancock MJ, French SD, Maher CG, et al. Effectiveness of interventions designed to reduce the use of imaging for low-back pain: a systematic review. CMAJ. 2015;187:401–8. https://doi.org/10.1503/cmaj.141183 .

Bennett S, Laver K, MacAndrew M, Beattie E, et al. Implementation of evidence-based, non-pharmacological interventions addressing behavior and psychological symptoms of dementia: a systematic review focused on implementation strategies. Int Psychogeriatr. 2021;33:947–75. https://doi.org/10.1017/S1041610220001702 .

Noonan VK, Wolfe DL, Thorogood NP, et al. Knowledge translation and implementation in spinal cord injury: a systematic review. Spinal Cord. 2014;52:578–87. https://doi.org/10.1038/sc.2014.62 .

Albrecht L, Archibald M, Snelgrove-Clarke E, et al. Systematic review of knowledge translation strategies to promote research uptake in child health settings. J Pediatr Nurs. 2016;31:235–54. https://doi.org/10.1016/j.pedn.2015.12.002 .

Campbell A, Louie-Poon S, Slater L, et al. Knowledge translation strategies used by healthcare professionals in child health settings: an updated systematic review. J Pediatr Nurs. 2019;47:114–20. https://doi.org/10.1016/j.pedn.2019.04.026 .

Bird ML, Miller T, Connell LA, et al. Moving stroke rehabilitation evidence into practice: a systematic review of randomized controlled trials. Clin Rehabil. 2019;33:1586–95. https://doi.org/10.1177/0269215519847253 .

Goorts K, Dizon J, Milanese S. The effectiveness of implementation strategies for promoting evidence informed interventions in allied healthcare: a systematic review. BMC Health Serv Res. 2021;21:1–11. https://doi.org/10.1186/s12913-021-06190-0 .

Zadro JR, O’Keeffe M, Allison JL, Lembke KA, et al. Effectiveness of implementation strategies to improve adherence of physical therapist treatment choices to clinical practice guidelines for musculoskeletal conditions: systematic review. Phys Ther. 2020;100:1516–41. https://doi.org/10.1093/ptj/pzaa101 .

Van der Veer SN, Jager KJ, Nache AM, et al. Translating knowledge on best practice into improving quality of RRT care: a systematic review of implementation strategies. Kidney Int. 2011;80:1021–34. https://doi.org/10.1038/ki.2011.222 .

Pawson R, Greenhalgh T, Harvey G, et al. Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10Suppl 1:21–34. https://doi.org/10.1258/1355819054308530 .

Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: illustrating the method for implementation research. Implementation Sci. 2012;7:1–10. https://doi.org/10.1186/1748-5908-7-33 .

Johnson MJ, May CR. Promoting professional behaviour change in healthcare: what interventions work, and why? A theory-led overview of systematic reviews. BMJ Open. 2015;5:e008592. https://doi.org/10.1136/bmjopen-2015-008592 .

Metz A, Jensen T, Farley A, Boaz A, et al. Is implementation research out of step with implementation practice? Pathways to effective implementation support over the last decade. Implement Res Pract. 2022;3:1–11. https://doi.org/10.1177/26334895221105585 .

May CR, Finch TL, Cornford J, Exley C, et al. Integrating telecare for chronic disease management in the community: What needs to be done? BMC Health Serv Res. 2011;11:1–11. https://doi.org/10.1186/1472-6963-11-131 .

Harvey G, Rycroft-Malone J, Seers K, Wilson P, et al. Connecting the science and practice of implementation – applying the lens of context to inform study design in implementation research. Front Health Serv. 2023;3:1–15. https://doi.org/10.3389/frhs.2023.1162762 .

Download references

Acknowledgements

The authors would like to thank Professor Kathryn Oliver for her support in the planning the review, Professor Steve Hanney for reading and commenting on the final manuscript and the staff at LSHTM library for their support in planning and conducting the literature search.

This study was supported by LSHTM’s Research England QR strategic priorities funding allocation and the National Institute for Health and Care Research (NIHR) Applied Research Collaboration South London (NIHR ARC South London) at King’s College Hospital NHS Foundation Trust. Grant number NIHR200152. The views expressed are those of the author(s) and not necessarily those of the NIHR, the Department of Health and Social Care or Research England.

Author information

Authors and affiliations.

Health and Social Care Workforce Research Unit, The Policy Institute, King’s College London, Virginia Woolf Building, 22 Kingsway, London, WC2B 6LE, UK

Annette Boaz

King’s Business School, King’s College London, 30 Aldwych, London, WC2B 4BG, UK

Juan Baeza & Alec Fraser

Federal University of Santa Catarina (UFSC), Campus Universitário Reitor João Davi Ferreira Lima, Florianópolis, SC, 88.040-900, Brazil

Erik Persson

You can also search for this author in PubMed   Google Scholar

Contributions

AB led the conceptual development and structure of the manuscript. EP conducted the searches and data extraction. All authors contributed to screening and quality appraisal. EP and AF wrote the first draft of the methods section. AB, JB and AF performed result synthesis and contributed to the analyses. AB wrote the first draft of the manuscript and incorporated feedback and revisions from all other authors. All authors revised and approved the final manuscript.

Corresponding author

Correspondence to Annette Boaz .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix a., additional file 2: appendix b., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Boaz, A., Baeza, J., Fraser, A. et al. ‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice. Implementation Sci 19 , 15 (2024). https://doi.org/10.1186/s13012-024-01337-z

Download citation

Received : 01 November 2023

Accepted : 05 January 2024

Published : 19 February 2024

DOI : https://doi.org/10.1186/s13012-024-01337-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Implementation
  • Interventions
  • Clinical practice
  • Research evidence
  • Multi-faceted

Implementation Science

ISSN: 1748-5908

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

an updated review of the literature

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Advanced Search
  • Journal List
  • Front Physiol

Whole-Body Cryotherapy in Athletes: From Therapy to Stimulation. An Updated Review of the Literature

Giovanni lombardi.

1 Laboratory of Experimental Biochemistry and Molecular Biology, I.R.C.C.S. Istituto Ortopedico Galeazzi, Milan, Italy

Ewa Ziemann

2 Department of Physiology and Pharmacology, Gdansk University of Physical Education and Sport, Gdansk, Poland

Giuseppe Banfi

3 Vita-Salute San Raffaele University, Milan, Italy

Nowadays, whole-body cryotherapy is a medical physical treatment widely used in sports medicine. Recovery from injuries (e.g., trauma, overuse) and after-season recovery are the main purposes for application. However, the most recent studies confirmed the anti-inflammatory, anti-analgesic, and anti-oxidant effects of this therapy by highlighting the underlying physiological responses. In addition to its therapeutic effects, whole-body cryotherapy has been demonstrated to be a preventive strategy against the deleterious effects of exercise-induced inflammation and soreness. Novel findings have stressed the importance of fat mass on cooling effectiveness and of the starting fitness level on the final result. Exposure to the cryotherapy somehow mimics exercise, since it affects myokines expression in an exercise-like fashion, thus opening another possible window on the therapeutic strategies for metabolic diseases such as obesity and type 2 diabetes. From a biochemical point of view, whole-body cryotherapy not always induces appreciable modifications, but the final clinical output (in terms of pain, soreness, stress, and post-exercise recovery) is very often improved compared to either the starting condition or the untreated matched group. Also, the number and the frequency of sessions that should be applied in order to obtain the best therapeutic results have been deeply investigated in the last years. In this article, we reviewed the most recent literature, from 2010 until present, in order to give the most updated insight into this therapeutic strategy, whose rapidly increasing use is not always based on scientific assumptions and safety standards.

Introduction

Local and systemic cold therapies (cryotherapies) are widely used to relieve symptoms of various diseases including inflammation, pain, muscle spasms, and swelling, especially chronic inflammatory ones, injuries, and overuse symptoms (Bettoni et al., 2013 ; Jastrzabek et al., 2013 ). The beneficial effects of cold as a therapeutic agent have been known for a long time, with ancient population aware about the reinvigorating effects of cold water either taken orally or used for baths. The use of cold, mainly locally, still remains in our daily common activities. A still up-to-date survey of a sample of Irish emergency physicians highlighted the fact that 73% of consultants frequently “prescribe” cold, 7% never suggest to use cryotherapy, and 30% is unsure about the benefits of using cold. Experience (47%) and common sense (27%) were the most frequently declared reasons for using ice, while only 17% referred to scientific reasoning (Collins, 2008 ).

Forty years ago, following personal observations of Prof. Toshiro Yamauchi (who recognized that the combination of cold and physical exercise was beneficial for clinical outcomes of treatments received by his patients', affected by rheumatoid arthritis, coming back from mountain localities after winter holidays), whole-body cryotherapy was introduced into clinical practice (Yamauchi et al., 1981a , b ).

At present, the use of very cold air in special, controlled chambers may be proposed for treating symptoms of various diseases (Bouzigon et al., 2016 ). Beside its clinical applications, a brief full body exposure to dry air at cryogenic temperatures lower than −110°C has become widely popular in sports medicine, often used to enhance recovery after injuries and to counteract inflammatory symptoms resulting from overuse or pathology (Furmanek et al., 2014 ). The number of studies about the use of whole-body cryotherapy (WBC) in sports medicine is growing, however, it is still lower than the topic's potential if the wide range of application of this methodology is considered. Studies published on athletes had mainly focused on post-training or competitive season recovery. Only a limited number of papers had investigated the effects of WBC used in preparation phase for competitive season to enhance form and performance, or during periods of high intensity of training to limit overuse and overreaching. Studies should be acknowledged to define safety, effectiveness, and efficacy of the treatment in athletes and to discover underlying molecular mechanisms supporting the claimed beneficial effects.

This review article collects the most recent literature (since 2010, Banfi et al., 2010b ) on whole-body cryotherapy with the purpose of delivering a complete and updated overview of the newest findings and the directions taken in research in this field. In particular, given the high number of new scientific findings mostly associated with great technological developments of this therapeutic method, this review discusses both technical aspects (i.e., therapeutic protocols, contraindications, thermoregulatory responses) and effects on a wide range of physiological (i.e., hematological, metabolic, energetic, endocrinological, skeletal, muscular, inflammatory) and functional parameters (post-exercise and post-traumatic recovery, pain, performance). We are aware of the limitations of this literature review. Almost all published research included in this review discuss results of using whole-body cryotherapy without providing any insight into molecular mechanisms involved in observed responses to the treatment. Also, although the review takes a non-systematic approach, an alternative meta-analysis would only offer a limited article coverage due to the type and, sometimes, the quality of available papers. Furthermore, we only reviewed reports on the WBC procedures performed in cryochambers (regardless of the cooling system, but considering the operating temperature); we do not consider treatments performed in cryosauna (also named cryocabins). Exposure to cold in a cryosauna cannot be deemed whole-body since during the treatment the head remains outside of the cabin. The two settings were concluded to, activate different molecular pathways and, possibly, exert different outcomes. Indeed, in a cryosauna, cooling is delivered through direct insufflation of liquid nitrogen vapors into the box. Free vapors are heavy and tend to remain within the cabin, below the chin; contrarily, in a nitrogen-cooled cryochamber liquid nitrogen fluxes through pipes inside the chamber's wall, and thus, there is no free nitrogen within the chamber. These differences also account for different safety standards of these treatments: free nitrogen vapor in a cryosauna could be potentially hazardous due to the risk of asphyxia.

In the present paper we refer to “whole-body cryotherapy,” which is the most commonly used term to define the methodology, but also to “whole-body cryostimulation,” which better describes effects of WBC in improving the metabolic and inflammatory responses as well as in enhancing recovery from exercise and injuries. In contrast, the term “cryotherapy” refers to a real therapy aimed at treating painful symptoms of inflammatory or traumatic conditions.

Technical aspects

Standardized protocol for wbc.

WBC is performed in special chambers, with the temperature and humidity strictly controlled. A subject, minimally dressed (for e.g., bathing suit, socks, clogs, headband, and surgical mask to avoid direct exhalation of humid air), enters a vestibule chamber at −60°C, where he stays for about 30 s of body adaptation and then passes to a cryochamber at −110° to −140°C, depending on the cooling system (electrical or nitrogen), where he remains for no more than 3 min. It is mandatory to remove any sweat before entry to avoid the risk of skin burning and necrosis. Access to the chamber is allowed only in the presence of a skilled personnel, controlling the procedures. A patients is free to leave the chamber at any time.

Contraindications

Being a medical therapy, WBC should follow strict guidelines and indications. Currently accepted contraindications for WBC include: cryoglobulinaemia, cold intolerance, Raynaud disease, hypothyroidism, acute respiratory system disorders, cardio-vascular system diseases (unstable angina pectoris, cardiac failure in III and IV stage according to NYHA), purulent-gangrenous cutaneous lesions, sympathetic nervous system neuropathies, local blood flow disorders, cachexia, and hypothermia, as well as claustrophobia and mental disorders hindering cooperation with patients during the treatment. When performed in the appropriate and controlled conditions, WBC is a safe procedure, which was demonstrated to be deleterious neither for lung (Smolander et al., 2006 ) nor heart function (Banfi et al., 2009a ); however, recorded observation of a very slight, clinically irrelevant increase in the systolic blood pressure (Lubkowska and Szygula, 2010 ) justifies precautions indicated for patients affected by cardiovascular conditions.

Temperature changes

Studying body temperature modifications following WBC, in comparison to changes observed in response to other cooling techniques, represents a hot topic. This is thought to be important since cooling effectiveness is the function of temperature decrease within a certain range.

Shifts in skin temperature (T sk ) of chosen body regions monitored by thermography and contact thermometry, before and immediately after a single WBC session (30 s at −60°C, 3 min at −120°C) showed, for the first time, the influence of body mass index (BMI) on the range of alternations. The highest magnitude of temperature changes was observed within lower extremities (tibias: −8.7°C; feet: −5.2°C), the mean total body temperature decreased by 5.8°C, while the internal body temperature dropped only by 0.8°C. The mean changes of temperatures at different sites correlated with BMI ( r = −0.46); for example, explicative images show that temperature decreased down to 8.1° and 7.9°C in a thin volunteer (BMI <25 kg/m 2 ) and down to 4.8° and 5.5°C in an obese participant (BMI > 30 kg/m 2 ), in the chest and back regions, respectively (Cholewka et al., 2012 ). Even more precisely than BMI, the fat-free mass index (FFMI: fat-free mass/height 2 ) and body fat percentage in males were both found to correlate with changes in skin temperature following WBC, (Hammond et al., 2014 ). Body composition was, thus, observed to be one of the main determinants of potential temperature changes and, possibly, of therapy's effectiveness. Cooling efficacy, indeed, differs between males and females as demonstrated by Hammond et al.; however, despite females having higher levels of adiposity than males, they experience greater mean temperature changes compared to males (12.07 ± 1.55°C vs. 10.12 ± 1.86°C). Compared to males, females have 20% smaller body mass, 14% more fat, 33% smaller lean body mass, and 18% smaller surface area, a higher subcutaneous to visceral fat ratio and a smaller ratio of fat mass index (FMI) to FFMI. Furthermore, females' BSA-to-mass ratio is higher than males, and the heat loss increases proportionally to this ratio. Under cold stress, females have a more extensively vasoconstricted periphery, with greater surface heat losses and show a significantly reduced sensitivity of the shivering response. Taken together these evidences could explain the discrepancy in cooling efficiency between sexes (Hammond et al., 2014 ).

Costello analyzed reduction in skin, muscle (vastus lateralis, at 1, 2, and 3 cm) and rectal temperatures following a single exposure to either WBC (−110°C) or cold-water immersion (CWI, at 8°C). Immediately after these procedures, the maximum drop in T sk was observed with WBC (−12.1 ± 1.0°C), marking a bigger drop compared to CWI (−8.8 ± 2.0°C). On the contrary, core (−0.3° to −0.4°C) and muscle (−1.2° to −2.0°C) temperatures shifted slightly with no differences between the two treatments and the maximum decrease occurring after 60 min (Costello et al., 2012b ). Similar results were obtained on changes in T sk at the patellar region; a greater drop was observed with WBC immediately after the procedure, while 10–60 min after the treatment a lower temperature was reached with CWI (Costello et al., 2014 ). Interestingly, the authors had set the question whether or not either WBC or CWI were capable of achieving the T sk (<13°C) believed to be required for analgesic purposes (Bleakley and Hopkins, 2010 ), yet they concluded that this temperature was reached by neither of the two procedures (Costello et al., 2014 ). Zalewski et al. confirmed that the maximum drop in core temperature occurred 50–60 min post-WBC (Zalewski et al., 2014 ).

In a systematic review, comparing 10 controlled trials, considering either a 10 min-long ice pack application, 5 min CWI, or 2.5–3 min WBC (−110° to −195°C), the authors illustrated that the largest reduction in T sk was obtained by the ice pack application due to the higher heat transfer constant ( k = 2.18) compared to water ( k = 0.58) and air ( k = 0.024). The obtained results confirmed negligible intramuscular temperature variation regardless of the cooling modality as well as importance of adiposity in determining cooling efficiency ( k = 0.23 vs. k = 0.46 of muscles; Bleakley et al., 2014 ).

In summary, the following reports have been made about the WBC treatment:

  • - WBC is a medical practice that must be performed in specialized facilities under supervision of a well-trained personnel.
  • - WBC has contraindications that must be considered before prescription.
  • - Cooling efficiency and, possibly, treatment effectiveness can be influenced by body composition.
  • - Due to differences in body composition, cooling efficiency is potentially greater in females than in males.
  • - WBC effectiveness in lowering T sk exceeds that of CWI; muscle and core temperatures seem to decrease in a similar way in response to both treatments.
  • - The maximum decrease in core temperature has been noted 50–60 min post-WBC.

The study of hematological response to WBC allows to define a wide range of effects covering modification of oxygen supply potential, inflammatory response, and coagulation function.

Erythrocytes and hemoglobin

We studied hematological parameters, including iron metabolism ones, in 27 athletes belonging to National Italian Rugby Team, during a summer camp (Lombardi et al., 2013a ). Two daily sessions of WBC (3 min, −140°C) were performed for seven consecutive days, one in the morning before the first training session, the second in the evening after the second training session. Athletes were strictly controlled for diet, especially the correct iron uptake. A typical plasma volume shift due to a prolonged training session of aerobic exercises was taken into account when interpreting the results. Among hematological parameters, erythrocytes (RBC), hematocrit (Ht), and hemoglobin (Hb) decreased noticeably; particularly, Hb decreased from 15.06 ± 0.84 to 14.70 ± 0.62 g/dL. Red cell distribution width (RDW) increased, indicating a rise of anisocytosis of RBC, although reticulocytes were stable, but the immature fraction of reticulocytes (IRF) was significantly decreased (Lombardi et al., 2013a ). A decrease of hemoglobinization could be a specific feature of the WBC treatment. Indeed, a similar decrease of Hb (about 0.3 g/dL) and IRF had been previously reported in rugby players, however, in that case, RBC and Ht had not been affected (Banfi et al., 2008 ). This difference could be attributable to a milder WBC protocol, with only five WBC (one per day, at −110°C). The decrease in the levels of Hb as well as RBC and Ht, is transitory and it recovered during continuative treatments as demonstrated by Szygula and colleagues in a study performed on students of the Polish National Military Academy, who can be considered physically active subjects, continuously performing exercises and controlled for variables as diet and lifestyle (Szygula et al., 2014 ). Recruited cadets were divided into two groups of 15 subjects; one group was treated with WBC, the other did not receive the treatment. Hematological parameters were measured after 10, 20, and 30 sessions, which were performed daily in a cryochamber at −130°C, for 3 min. After 10 sessions, Hb decreased from a mean of 15.1 ± 0.74–14.4 ± 0.94 g/dL and remained at this concentration after 20 sessions (14.5 ± 0.71 g/dL). It then rose to 15.1 ± 1.1 g/dL after 30 sessions. Similar changes were observed for Ht and RBC. The decrease of Hb, RBC and Ht lasted through 20 sessions of the WBC treatment; then the bone marrow reacted by releasing new RBCs (Szygula et al., 2014 ). A decrease in Hb and RBC was already described in elite Polish field hockey players after 18 sessions of WBC (Straburzyńska-Lupa et al., 2007 ). Hb also showed a decreasing though not statistically significant trend, dropping from 15.0 ± 1.0 to 14.4 ± 0.8 g/dL, in nine collegiate physically active subjects, who had completed 30 min step up/down exercise, aimed at inducing delayed-onset muscle soreness (DOMS), and had been treated with two daily WBC sessions for 5 consecutive days. In opposite, the control group, which had undergone the same DOMS-inducing training without the WBC or any other recovery treatment, experienced stable levels of Hb (Ziemann et al., 2014 ). Nevertheless, some data revealed that Hb and RBC were stable in 12 professional tennis players, following 10 sessions of WBC applied twice a day, at –120°C for 3 min, over 5 days, during a controlled training camp (Ziemann et al., 2012 ) as well as in 16 kayakers treated twice a day for the first 10 days of a 19 day physical training cycle (Sutkowy et al., 2014 ). It is thus, possible that shifts in Hb and RBC induced by WBC are dependent on the discipline and baseline hematological profile. This issue, however, still has not been investigated. Mean curpuscular volume (MCV) grew following the WBC treatment applied in rugby players and in field hockey players (Straburzyńska-Lupa et al., 2007 ; Lombardi et al., 2013a ); in the latter group values of MCV, mean curpuscular hemoglobin (MCH), and of mean curpuscular hemoglobin concentration (MCHC) remained elevated up to a week after the end of the treatment (Straburzyńska-Lupa et al., 2007 ).

A slight dehemoglobinazion has two direct consequences. Firstly, since the OFF-score, a parameter used to calculate the probability of blood doping in athletes, depends on Hb concentration and Ret count (Sottas et al., 2010 ; Robinson et al., 2011 ; which remained stable), WBC may reduce the result of this score and, thus, cannot be considered a performance enhancing practice. On the other hand, the use of WBC to mask illicit practices is unjustified because the potential decrease in Hb is too small and the change itself is short-lasting and/or temporary (Lombardi et al., 2013a ). Secondly, the decrease in Hb and RBC should be considered when the timeline of recovery strategies, within a competitive season, is drawn.

Iron metabolism

Martial status was not modified after the treatment in 27 rugby players submitted to two daily WBC sessions for 7 consecutive days (Lombardi et al., 2013a ). Only soluble transferring receptor (sTfR) increased significantly, but not pathologically, possibly demonstrating initial high functional iron demand (Lombardi et al., 2013b ). Similar results were obtained in a more recent paper by Dulian and colleagues. Regardless of the fitness level, in a cohort of obese subjects (BMI > 30 kg/m 2 ), serum iron and ferritin remained unchanged after the 1st and 10th WBC session. Only hepcidin, a hepatocyte-derive peptide hormone mediating iron depletion in inflammation (Lombardi et al., 2013b ), decreased moderately (Dulian et al., 2015 ).

WBC enhances hemolysis, which could explain the Hb decrease during initial phase of the treatment. A decrease of haptoglobin, scavenger protein for free Hb released from broken RBC was described in the above-mentioned paper by Szygula and co-workers, after 10 and 20 WBC sessions, but a recovery appeared after 30 sessions, following the changes in Hb and RBC. Contemporarily, bilirubin increased, reflecting Hb catabolism. Hemolysis stimulated release of erythropoietin (EPO), which increased by 4.5% compared to baseline after 10 sessions, and further by 10.8 and 10.1% after 20 and 30 sessions, respectively, possibly supporting the recovery of RBC number after the initial decrease. Even in the case of EPO, the shifts in concentrations remained within physiological ranges (Szygula et al., 2014 ).

Levels of leukocytes did not show any changes after 14 sessions of WBC (twice a day, over 7 days) in the group of 27 rugby players, belonging to National Italian Rugby Team, studied during a summer camp (Lombardi et al., 2013a ). The same was found for the group of 16 kayakers treated twice a day for the first 10 days of a 19 day physical training cycle (Sutkowy et al., 2014 ).

At the same time, leukocytes increased in the students of the Polish Military Academy after 10 and 20 sessions, but returned to baseline values after 30 sessions. The increase trend covered both granulocytes and lymphocytes (Szygula et al., 2014 ). Similar increase was also reported in tennis players, but not for subcategories of granulocytes and lymphocytes (Ziemann et al., 2012 ). Despite the increase, leukocytes always remained within the physiological range. Mobilization of leukocytes from the bone marrow and organs of residence has been hypothesized as a possible cause of these increases although an explanation of this phenomenon is still lacking.

In endurance trained runners, a simulated 45 min trail run, designed specifically to trigger exercise-induced muscle damage (EIMD), followed by four sessions of WBC applied once a day, resulted in an increase in neutrophil count of 114% compared to baseline, with the maximum peak recorded 1 h after the exercise. The correspondent increase in neutrophils, following passive recovery, accounted for 101% shift against baseline. The authors hypothesized that the increase of circulating neutrophils stimulated angiogenesis (via vascular endothelial growth factor—VEGF expression) and the consequent improved perfusion was associated with a reduced delayed onset of muscle soreness (DOMS) and, hence, an improved recovery (Pournot et al., 2011 ).

Platelets did not shift in response to WBC sessions applied in groups of rugby and tennis players (Lombardi et al., 2013a ; Ziemann et al., 2014 ) nor students of the Polish Military Academy (Szygula et al., 2014 ).

  • - WBC causes a decrease in Hb, Ht, and RBC after 5, 10, and 20 sessions. A recovery of hemoglobinization is reached after 30 sessions. Ret counts remains unaffected by WBC.
  • - The effect of WBC on RBC and Hb can be influenced by the type and intensity of physical training since in some groups of athletes these changes did not occur.
  • - Hemolysis may be the cause behind the drop in RBC, Hb, and Ht following the WBC treatment of 10–20 sessions.
  • - EPO is induced in the course of WBC with the aim to recover to baseline levels of RBC and Hb.
  • - WBC should not have a boosting effect on bone marrow and is not influencing athletes' hematological parameters usually controlled to test for illicit bone marrow stimulation.
  • - The level of leukocytes either does not change or only slightly increases in response to WBC. Cryotherapy possibly mobilizes leukocytes, especially neutrophils, with a positive effect on DOMS.
  • - Platelets are not affected by WBC.

Lipids concentrations and energy metabolism

Lipids are the main source of energy as well as the main thermogenic substrate. It is thus, possible that an intense cold stimulus of WBC affects lipid metabolism.

Sixty-nine physically active male subjects (10 professional rugby players and 39 healthy individuals including policemen and soldiers) were divided into three groups based on the number of WBC sessions (−130°C, 3 min): (i) 5, (ii) 10, and (iii) 20. Five sessions of WBC did not modify the lipid profile. Ten sessions induced a 34% decrease in triglycerides and much more after 20 sessions (from 108 ± 50 mg/dL, before the start of the treatment, to 69.4 ± 27.2 mg/dL, after 20 sessions). After 20 sessions, high density lipoprotein (HDL) cholesterol significantly increased (from 53.2 ± 16.5 to 63.1 ± 27.4 mg/dL), whilst low density lipoprotein (LDL) cholesterol decreased noticeably (from 97.7 ± 48.3 to 72.8 ± 52.0 mg/dL; Lubkowska et al., 2010a ). Concentrations of total cholesterol and LDL also decreased in physically active males, who underwent a 30 min step up/down exercise supplemented with WBC applied twice a day over 5 consecutive days; compared to the control group, total cholesterol and LDL dropped by 43 and 52%, respectively (Ziemann et al., 2014 ).

Adipose tissues, both white (WAT) and brown (BAT), become activated during cold exposure. BAT is particularly consumed during exposure to cold, contributing to energy metabolism. However, 6 months of moderate aerobic activity combined with WBC did not change body mass, fat, or lean body mass percentages. Circulating adiponectin, leptin, and resistin concentrations were also not modified while, again, LDL and triglycerides decreased and HDL increased. The experiment was performed on 45 overweight and obese men, asked to complete a test on a bicycle ergometer, beginning with a workload of 1 W/Kg of fat-free mass, then increased by 0.5 W/Kg half until exhaustion. Participants exercised three times a week under researchers' supervision; once a week they completed a 45 min-long walking session and, twice a week, attended a 45 min gym session. After 1 month from the start of the fitness program and 1 month before its end, 20 daily WBC treatments at −130°C for 3 min were done (Lubkowska et al., 2015 ).

Ten WBC sessions (−120°C, 3 min), applied twice day over 5 days, in 12 professional tennis players, during a controlled training camp, affected neither the resting metabolic rate nor the percentage of fat used as a metabolic substrate; this was also the case with healthy, physically active men treated with two daily WBC sessions, for 5 consecutive days between step up/down 30 min-long exercises (Ziemann et al., 2012 , 2014 ).

Recently, a differential effectiveness of WBC has been implied to depend on the starting fitness status of the subject. Middle-aged obese men (BMI > 30 kg/m 2 ) were grouped based on their fitness level (low and high cardiorespiratory fitness level) and exposed to 10 consecutive sessions of WBC over 2 weeks. Irisin, a myokine induced by exercise stimulating WAT browning, linked to enhanced thermogenic capacity (Boström et al., 2012 ; Lombardi et al., 2016 ), increased slightly in both groups over the course of 24 h after the first session of WBC. More interestingly, irisin plasma concentrations increased by 20% in subjects with a low fitness level, but decreased slightly in subjects with a high fitness level (Dulian et al., 2015 ). The observed increase in irisin corresponds with data of Lee and co-workers, who investigated changes of myokine—irisin and adipokine—fibroblast growth factor 21 (FGF21) in response to cold water immersion. In contrast to trends in irisin, they observed a greater reduction in FGF21, interpreted as a pronounced effect of the non-shivering thermogenesis (Lee et al., 2014 ). There are no data about changes in FGF21 in response to WBC.

  • - WBC has a dose-dependent improving effect on the lipid profile.
  • - WBC does not affect the rest metabolic rate and energy expenditure during exercise.
  • - WBC has a stimulatory effect on irisin expression, which should act at the adipose tissue level by enhancing thermogenesis.

Bone metabolism and skeletal health

Bone health is essential not only in whole-body health, but also in determining and sustaining athletes' performance (Lombardi et al., 2016 ; Sansoni et al., 2017 ). Due to high energy expenses associated with high-level physical activity (Lombardi et al., 2012 ), the risk of osteopenia and stress fractures (the inability of the skeleton to modify its own microarchitecture depending on applied loads) is always present. Based on these findings, the availability of medical tools favoring recovery of the bone tissue is highly desirable (Banfi et al., 2010a ; Lombardi et al., 2011 ).

Ten professional rugby players, belonging to Italian National Team, submitted to single daily sessions of WBC for 5 consecutive days (−110°C, 2 min), were compared to 10 players, who completed the same training protocol without WBC (Galliera et al., 2012 ). Bone metabolism was studied through biochemical parameters. The soluble ligand of the receptor activator of nuclear factor κB (RANKL) and its decoy receptor osteoprotegerin (OPG) constitute a fundamental cytokine system connecting the immune system and bone metabolism in order to link pro- and anti-inflammatory balance to calcium stores. RANKL is released from osteoblasts and lymphocytes and activates osteoclasts, inducing bone resorption. Osteoclasts express RANK, specific receptor of RANKL, which induces an intracellular signal to reabsorb bone. The effect of RANKL on RANK is blocked by OPG (Banfi et al., 2010a ). WBC did not affect plasma RANK and RANKL concentrations in athletes, but increased OPG, thus, causing the OPG/RANKL ratio, an index of resorption-to-formation balance, to grow as well. The increased osteogenic potential may have a role in the post-fracture recovery, but also in prevention of more insidious stress fractures (Galliera et al., 2012 ).

  • - WBC could counteract the inflammation-induced bone resorption.

Inflammatory markers

The anti-inflammatory and analgesic effects of cryotherapy are the most searched for by athletes and patients. The effectiveness of the treatment in this specific ambit has been demonstrated by a number of studies although the available data is contrasting and still debated. Indeed, along with studies reporting direct effect of WBC on inflammatory markers, there are several reports that did not link any changes with the treatment at all. Nevertheless, almost all of these studies agree on general benefits induced by the treatment including improved pain, mood, and quality of life (QoL). A prototypic example was found in patients affected by fibromyalgia, an auto-inflammatory disease of unknown pathogenesis and highly variable clinical presentation, always characterized by chronic systemic inflammation, generalized pain, and severe fatigue, which invariably and considerably deteriorates the QoL; regardless of the conventional analgesic therapy, compared to subjects not treated with WBC ( n = 50), patients receiving the WBC treatment ( n = 50) exhibited large clinical improvements in all the investigated parameters depicting the QoL and the ability to perform daily activities [pain by visual analog scale (VAS), global health status (GH), fatigue severity scale (FSS), and short form (SH)-36; Bettoni et al., 2013 ].

Tumor necrosis factor (TNFα), interleukin 6 (IL-6), and IL-10 did not change in 11 runners, who underwent a 48-min pro-EIMD training protocol and were treated four times with either WBC at −110°C or passive recovery. On the contrary, the increase in the pro-inflammatory marker IL-1β 24 h post treatment and C-reactive protein (CRP) 96 h post-WBC, were strongly attenuated compared to passive recovery. In parallel, WBC had a greater inducing effect on the anti-inflammatory IL-1 receptor antagonist (IL-1ra) 1 h post-treatment. The authors concluded that a unique session of WBC (3 min at −110°C), performed immediately after exercise, enhanced muscular recovery by restricting the inflammatory process (Pournot et al., 2011 ). Five days of WBC applied twice a day (−120°C, 3 min), combined with moderate-intensity training, in six professional tennis players, during a controlled training camp, decreased TNFα serum concentrations by 60% as opposed to 35% decrease observed in the untreated group. IL-6, instead, increased slightly, but significantly due to WBC (Ziemann et al., 2012 ). The pro-inflammatory role of IL-6 has been recently revised (Peake et al., 2015 ). Indeed, it has a dual effect depending on the production site and its basal concentration. Chronically high, even if moderately, IL-6, characterizing chronic inflammatory conditions (e.g., obesity, sedentariness) stimulating the hepatic synthesis of this cytokine, cause pro-inflammatory, and potentially deleterious effect. On the contrary, even very high, pulsatile spikes of IL-6, originating from low basal concentrations, derived from contracting muscles, act as a powerful anti-inflammatory mediator (Lombardi et al., 2016 ). In 18 professional male volleyball players, a session of submaximal exercise increased IL-1β and IL-6 by 60%. The WBC treatment (−130°C, 3 min), however, performed before the exercise, prevented these increases. This study, thus, highlighted a possible preventive effect of WBC on exercise-induced inflammation (Mila-Kierzenkowska et al., 2013 ). However, in rugby players, a single session of WBC (−130°C) did not affect IL-6 values, regardless of the treatment‘s duration (1, 2, or 3 min; Selfe et al., 2014 ). The protective effect of WBC was also demonstrated in 18 physically active college-aged men, who underwent eccentric workout inducing DOMS. A single session of WBC (−110°C, 3 min) performed after the exercise had the same effect of a passive recovery (increased IL-6, IL-1β, and IL-10). After 5 days, the repeated performed exercise induced the similar biochemical modifications in the untreated group, while in the WBC-treated group (5 days, twice daily) IL-6 and IL-1β were unchanged compared to baseline with IL-10 strongly elevated (Ziemann et al., 2014 ).

In healthy young men, the extent of IL-6 increase, 30 min and 24 h after a single session of WBC, was much greater than the corresponding increases observed after 10 days of the treatment (Lubkowska et al., 2010b ). Interestingly, the same authors also demonstrated that in 45 healthy men, a different number of sessions had a different effect on particular inflammatory parameters. Five sessions of WBC increased IL-10 by 30%, but this change was dissipated 2 weeks after the end of the treatment. After 10 sessions IL-1α decreased by 17%, while IL-6 and IL-10 increased by 10% and 14% respectively, yet even in this case, these modifications were lost after 2 weeks. After 20 sessions of WBC the cytokine balance was confirmed just like after 10 sessions, but in this case, the decrease in IL-1α was sustained 2 weeks after the end of the treatment. IL-1β, TNFα, and IL-12 remained unchanged over the observation period. Consequently, the authors suggested to use 20 sessions in order to induce adaptation (Lubkowska et al., 2011 ). In professional rugby players, a slight, yet not significant decrease in ferritin and transferrin concentrations was observed that, together with an increase in the sTfR concentration, further supported the anti-inflammatory effect of WBC. Indeed, these parameters generally have a contrary behavior, when a chronic inflammatory process is ongoing (Lombardi et al., 2013b ). Interestingly, as for hematological parameters and martial status, a shift in the cytokine profile depends upon fitness capacity. Pro-inflammatory cytokines (IL-6, TNFα) and adipokines (resistin, visfatin) were higher in obese LCF subjects compared to the obese HCF ones. IL-10 increased equally in both groups, with adiponectin and leptin unaffected (Ziemann et al., 2013 ). Even the drop in CRP, observed regardless of the number of sessions (either 1 or 10), was much greater in obese LFL than in obese HFL (Dulian et al., 2015 ). The anti-inflammatory effect of WBC is in line with previous reports (Banfi et al., 2009b ).

Table ​ Table1 1 presents the findings about the effects of WBC on circulating levels of cytokines and inflammatory markers.

Effects of WBC on circulating levels of cytokines and inflammatory markers .

It is worth noting that cooling therapies are often applied interchangeably in athletes' recovery treatment. Reducing inflammation may have another interesting aspect when achieved in athletes. The latest paper published by Roberts and co-workers revealed that a cooling therapy (cold water immersion-CWI: 10 min using a circulatory cooling unit, temperature at 10.1 ± 0.3°C) used after resistance training as a recovery strategy attenuates both acute changes in satellite cell numbers and kinase activity that regulates muscle hypertrophy, which may translate into smaller, but longer-term training gains in muscle strength and hypertrophy. Consequently, reduced inflammation in response to WBC may have a negative effect on muscle hypertrophy (Roberts et al., 2015 ).

  • - WBC induces anti-inflammatory effects.
  • - Findings about the effect of WBC on IL-6 are not always concordant, probably due to differences in exercise protocols applied. In general terms, a single session of WBC increases IL-6 concentration, while multiple sessions recover it to baseline. In this sense, WBC seems to mimic exercise-induced impacts.
  • - More consistently, WBC stimulates the anti-inflammatory response (reduced IL-1β and increased IL-10, IL-1Ra).
  • - Fitness capacity affects the inflammatory response to WBC in non-athletes.
  • - Further, investigations should focus on establishing, whether reduced inflammation has a beneficial effect on athletes' performance if they combine training and cold therapy.

Endocrine function and hormone profile

Monitoring hormones is essential to assess health status especially in athletes, who experience chronic intense workloads associated with psychophysically stressful situations. WBC is used to relieve stress conditions owing to the activation of neuroendocrine and metabolic axes regulating thermal homeostasis. However, only a few of reports published so far had considered this aspect.

Salivary steroid hormones were monitored in 25 professional top level rugby players, during a summer training camp. The athletes were submitted to the WBC treatment (−140°C, 3 min) twice a day for 7 consecutive days, the first before the morning training session, the second after the evening workout. Saliva was collected before the start of the camp, after the evening WBC session on the first day, and after the last WBC session on the last day. Compared to baseline, at the end of the first day (2 WBC sessions completed) cortisol and dehydroepiandrosterone (DHEA) decreased. After 7 days, cortisol, DHEA, and estradiol decreased, yet testosterone increased. Importantly, in the majority of subjects variations exceeded the critical difference (CD). CD is the minimal value, calculated between two consecutive measurements of the same parameter, using the same method, on the same individual and including analytical and biological variabilities, which when exceeded, testifies that an external factor is really modifying the investigated parameter. Notably, the testosterone-to-cortisol ratio, widely accepted as an index of the potential athletic performance status, increased as a result of the observed changes (Grasso et al., 2014 ).

Cortisol increased in six professional tennis players treated with WBC twice a day for 5 days (−120°C, 3 min) after moderate-intensity training, during a controlled training camp (Ziemann et al., 2012 ), while testosterone remained stable. Cortisol and testosterone were both stable in 16 kayakers from Polish National Team, who combined exercise with two daily WBC sessions during the first 10 days of a training cycle in preparation for the World Championships (Sutkowy et al., 2014 ). Six elite rowers were subject to a 6 day training cycle combined with training sessions scheduled twice daily, each preceded by WBC (−125/−150°C, 3 min). During a control training session without WBC, subjects experienced an increase in cortisol on the 3rd day of training; WBC delayed and reduced this increase on the 6th day (Wozniak et al., 2013 ). After a normal training week, 10 elite synchronized swimmers performed two 2 week sessions of an intensified training, randomly combined either with or without daily WBC. The two sessions were separated by a 9 day light training period. Although salivary cortisol was unchanged, WBC mitigated signs of functional overreaching observed during the control period such as reduced sleep quantity, increased fatigue and impaired exercise capacity (Schaal et al., 2015 ).

A randomized, counterbalanced and crossover study on 14 habituated English Premier League academy soccer players, a single WBC session (−135°C, 2 min), performed within 20 min after a repeated sprint exercise (15 × 30 m), increased salivary testosterone up to 24 h post-exercise. Still, the treatment had no effect on cortisol, blood lactate, and CK (Russell et al., 2017 ).

  • - WBC affects the hormonal asset, decreasing hormones typically associated with psychophysical stress, such as cortisol, and increasing testosterone, a typical anabolic hormone.
  • - Cortisol modifications during WBC treatment cycles are not consistent in the current literature, possibly owing to different stressors applied.

Redox balance

Oxidative stress is the main factor affecting athletes' performance. Muscle activity generates oxidants released in the intercellular space following membrane leakage or breaking. Oxidants activate chain reactions, amplifying production of reactive oxygen species (ROS), which damage membranes, cellular structures and DNA. Inflammation activates innate immunity, which produces ROS and, through a vicious cycle, ROS-sustained inflammation (Slattery et al., 2015 ). WBC has been advocated to possibly enhance antioxidant capacities and, thus, counteract the exercise-induced ROS production.

The study on WBC effects on an oxidant-antioxidant balance was performed in 16 kayakers belonging to Polish National Team submitted to a 19 day physical training cycle. During the first 10 days of the training cycle, the kayakers combined exercise with two daily WBC sessions: the first in the morning before exercise, the second session in the afternoon after exercise. Day by day, the temperature was decreased from −120 to −145°C. Blood drawings were performed at the beginning of the training period as well as after 5, 11, and 19 days. Lipid peroxidation products malondialdehyde (MDA), conjugate dienes, and thiobarbituric acid reactive substances (TBARS) did not show any changes. Glutathione peroxidase (GPx) activity decreased after 5 days, yet later it increased again to baseline following WBC (11th day). On the last day of training, 9 days after the end of the WBC treatment, TBARS decreased compared to baseline and the level on day 5, whilst GPx increased compared to day 5. WBC may improve the efficiency of TBARS elimination that positively impacts strenuous exercise (Sutkowy et al., 2014 ).

Eighteen professional volleyball players performed the first 40 min submaximal exercise bout on an ergometer 2 weeks after the end of season, and the second bout 2 weeks after the first one. A single WBC treatment (−130°C) was conducted before the first exercise bout. The activity of RBC catalase after WBC + exercise was two times lower compared to that recorded after control exercise, without WBC, as was the activity of RBC superoxide dismutase (SOD; −8%; Mila-Kierzenkowska et al., 2013 ).

SOD and GPx activities were lower (by 44 and 42%, respectively) after 3 days of training combined with WBC than without the treatment in six elite rowers, subject to a 6 day training cycle. Lower levels of TBARS and conjugated dienes in erythrocytes were also found in the WBC-treated group (Wozniak et al., 2013 ).

In healthy subjects, 24 males and 22 females, after 10 daily WBC sessions, plasma uric acid, SOD, and total antioxidants all increased compared to both baseline and the control group not submitted to WBC. TBARS, although unchanged compared to baseline, were also higher than in the control group (Miller et al., 2012 ). During a 6 month long physical exercise program with two daily WBC exposures over 20 sessions, a significant decrease in SOD, catalase and glutathione reductase activities was observed (13, 8, and 70%, respectively). SOD activity increased after successive WBC sessions, while catalase activity progressively decreased (Lubkowska et al., 2015 ).

Although a general beneficial antioxidant effect already after a limited number of WBC sessions is described in these studies, a dose-dependency was observed in healthy men, with 20 sessions being the optimal number (Lubkowska et al., 2012 ).

  • - WBC has a dose-dependent improving effect on the redox balance after exercise.
  • - WBC is able to decrease the activity of RBC enzymes probably as a reaction to a decrease in total oxidants and an increase in antioxidants.

Muscle damage parameters, fatigue recovery, and pain

From athletes' and sports physicians' point of view the effect of WBC on muscle damage and recovery is the most important one. Indeed, muscle recovery has been the most claimed and, possibly, the most debated effect of WBC. In this specific field, the stimulatory effect of WBC is particularly claimed.

EIMD was studied in nine endurance runners, comparing three different recovery modalities: WBC, far-infrared (FIR), and passive. The runners performed three identical repetitions of a simulated trail run on a motorized treadmill. Recovery was evaluated 1, 24, and 48 h post-exercise. The eventual decrease in maximal isometric force, after three isometric voluntary contractions of knee extensor muscles, was used for judging muscle damage due to strenuous exercise. The best results were observed after WBC at 1, 24, and 48 h post-exercise. WBC-enhanced psychological recovery within days after the exercise including decreased perception of muscular tiredness and pain, already after the first session of WBC. At the same time the pain sensation was lowered by FIR only 48 h after the exercise and it did not change at all as a result of passive recovery. Well-being, evaluated through a standardized questionnaire for tiredness and pain, was improved after 24 h due to WBC and after 48 h due to FIR. Still, an increase in serum activity of creatine kinase (CK), typical of strenuous exercise, was not improved by three WBC sessions (Hausswirth et al., 2011 ).

A 40% decrease in CK activity was reported in rugby players after five consecutive daily sessions of WBC (Banfi et al., 2009b ); it also decreased by 34% after 10 sessions in kayakers (Wozniak et al., 2007 ). The same decreasing trend of CK was also confirmed in 12 professional tennis players (CK dropped from 305.0 to 241.4 U/L in six treated subjects, while remained unchanged in the control group ones: 286.7 and 295.5 U/L), during a controlled training camp, during which players were treated with WBC twice a day for 5 days (−120°C, 3 min; Ziemann et al., 2012 ). The same was also noted in six elite rowers, subject to two 6 day training cycles with training sessions twice a day, all either preceded by WBC or not (Wozniak et al., 2013 ) as well as in physically active males treated with two daily WBC sessions for 5 consecutive days between step up/down 30 min exercises; in the latter case, the changes were also accompanied by a significantly improved pain perception (Ziemann et al., 2014 ). In professional rugby players, after a 7 day training camp with two daily WBC sessions, lactate dehydrogenase (LDH), and aspartate aminotransferase (AST) activities increased as a consequence of high workload, whilst CK decreased slightly, but significantly. Moreover, kidney function (estimated glomerular filtration rate, eGFR), which could be impaired following muscle damages, was unaffected by the treatment (Lombardi et al., 2014 ).

The WBC-induced attenuation in serum soluble intercellular adhesion molecule (sICAM)-1 immediately after EIMD may be responsible for reduced acute inflammatory response muscle damage (Ferreira-Junior et al., 2014 ). The level of sICAM-1 together with CK and LDH activities decreased in rugby players after 5 consecutive days of daily WBC sessions (Banfi et al., 2009b ). After muscle damage is induced by exercise, leukocytes are mobilized to injured tissues by sICAM-1 intervention, where pro-inflammatory cytokines and ROS are released. WBC causes vasoconstriction and reduces the number of leukocytes reaching the muscles by inhibiting sICAM-1 (Ferreira-Junior et al., 2014 ). However, it was also hypothesized that cold could induce an extra-release of sICAM1 and, therefore, cause a decrease of muscle inflammation; the final net effect is the same (Dugué, 2015 ).

Beneficial effects of WBC on psychological recovery within days after exercise included a decreased perception of muscular tiredness and pain- an greater-improvement compared to the effect of other recovery modes, namely FIR and passive (Pournot et al., 2011 ). Daily WBC in 10 top level female synchronized swimmers (−110°C, 3 min), applied following training sessions, improved athletes' tolerance to training load by preserving sleep quantity during the period of intensive training before the Olympic Games (Schaal et al., 2015 ). The effect was evaluated after the athletes had been randomly assigned either WBC or non-WBC supported recovery. WBC use had a beneficial effect on sleep duration, limiting sleep latency possibly conditioned by post-exercise parasympathetic reactivation (Schaal et al., 2015 ).

On the contrary, in a randomized, counterbalanced and crossover design, 14 habituated English Premier League academy soccer players, Russell and colleagues found that a single WBC session (−135°C, 2 min), performed within 20 min after a repeated sprint exercise (15 × 30 m), increased salivary testosterone, yet had no effect on cortisol, blood lactate, and CK nor on performance (peak power output), recovery, and soreness perceptions (Russell et al., 2017 ).

It is worth noting that, most of the data reviewed had been obtained from athletes off season or during preparation training phase. It mostly come from endurance athletes or from athletes with a dominating aerobic metabolism. Limited data exist on the conjunction of resistance training and WBC.

  • - WBC could limit the release of intracellular enzymes, but only after a prolonged cycle of consecutive sessions.
  • - WBC-associated improvements in muscular tiredness, pain, and well-being after strenuous exercise have been reported in the majority, but not all, of the reviewed studies.
  • - WBC-mediated enhancement of muscular recovery depends on the limitation of the exercise-induced inflammatory response.

Performance recovery

Performance recovery using different cooling methods, especially CWI and contrast water immersion, has been extensively studied so far. Their average effect on recovery of trained athletes is rather limited, as reported in a recent review, but under appropriate conditions (whole-body cooling, recovery from sprint exercise) post-exercise cooling has positive effects even for elite athletes (Poppendieck et al., 2013 ).

Positive effects induced by WBC after 96 h were reported in 18 physically active subjects, who performed a single maximal eccentric contractions of the left knee extensors, through two WBC sessions (−110°C) 24 and 48 h after exercise. The effects were negative at 24 and 48 h post-exercise (Costello et al., 2012a ). Positive effects were also reported 24 and 48 h after the treatment in nine runners completing a simulated 48-min trail run, submitted to three WBC sessions, immediately after the exercise as well as 1 and 2 days after (Hausswirth et al., 2011 ).

Eleven endurance athletes were tested twice in a randomized crossover design with 5 × 5 min of high intensity running followed by 1 h of passive recovery, including either WBC (−110°C, 3 min) or a 3 min walk. Time-to-exhaustion difference between a ramp-test protocol before running and 1 h post-recovery was lower in WBC-treated subjects. WBC improves acute recovery during high-intensity intermittent exercise in thermoneutral conditions. This could be induced by enhanced oxygenation of the working muscles as well as by reduction of cardiovascular strain and increased work economy at submaximal intensities (Krüger et al., 2015 ). In addition to beneficial effects on inflammation and muscle damage, WBC induces peripheral vasoconstriction, which improves muscle oxygenation (Hornery et al., 2005 ), lowers submaximal heart rate and increases stroke volume (Zalewski et al., 2014 ), stimulates autonomic nervous parasympathetic activity and increases norepinephrine (Hausswirth et al., 2013 ). These effects favor post-exercise recovery and induce analgesia (Krüger et al., 2015 ).

Although these evidences, a recent meta-analysis by Bleakley et al., based on a small number of randomized studies, highlighted that WBC sustains improvements in subjective recovery and muscle soreness following metabolic or mechanical overload, but little benefit toward functional recovery (Bleakley et al., 2014 ). The authors concluded that, until further researches will be available, less expensive cooling modality (local ice-pack, cold water immersion) would be used in order to gain the same physiological and clinical effects to WBC.

Three-minute WBC exposure significantly differ from a 1–2-min exposure. Blood volume decreased within vastus lateralis and gastrocnemius occurred 0–5 min after WBC in 14 professional rugby players. Oxyhemoglobin and deoxyhemoglobin increased in 15 min post-WBC, reaching baseline values indicative of venous pooling. Extreme cold induces vasodilation after constriction in very short time. Gastrocnemius is more susceptible to pooling at all exposure times than vastus lateralis. Two-minute WBC exposure causes changes in core and T sk , tissue oxygenation in vastus lateralis, and gastrocnemius and thermal sensation. The optimum exposure time is 30 s at −60°C followed by 2 min WBC at −135°C (Selfe et al., 2014 ).

It is also crucial to keep a constant temperature between two consecutive treatments. Door opening and subject permanence within a chamber increase temperature and reduce therapeutic effectiveness, particularly for electrical cryochambers, but also for liquid nitrogen-cooled chambers. A 2 min wait between two consecutive treatments would allow temperature recovery to therapeutic levels.

The number of sessions is crucial for WBC effectiveness, as previously discussed. A recent Cochrane review, reporting on the absence of beneficial effects of WBC on prevention and treatment of muscle soreness in athletes, involves on only four papers. One out of these four papers talked about six treatments in cryocabin, the other two investigated the effects of a single treatment in a cryochamber and the final one reported the effects of only three treatments in a cryochamber (Costello et al., 2015 ). A single session is probably not sufficient to exert any significant effect. Twenty consecutive sessions should be a minimum for effectiveness evaluation; 30 sessions should be the optimum, because a complete hematological and immunological recovery after the initial response is possible (Szygula et al., 2014 ). Studies evaluating long-term WBC treatment are not easily performable in professional athletes during competitive seasons, but they could be proposed during training and summer camps. Although offseason injuries are rarer than contusions incurred during competitions, it is important to note that standardization of exercise and training offseason is more easily achievable.

Furthermore, randomization is very difficult, if not impossible, to be proposed to elite athletes, and professional teams: the treatment is proposed to improve recovery or to prevent injuries, thus, it should not be limited to a subgroup of athletes. On the other hand, when WBC is used for accelerating recovery from trauma/injury, only injured athletes are treated. Crossover studies could be more easily performed during training camps (but not during competitive season), but they would be only devoted to physiological modifications and not to recovery.

Different, and sometime discrepant results presented in current literature could be attributable to different levels of subjects ranging from “physically active” to “elite” to “national/international selection.” A stratification of WBC effects should be evoked for different subjects, because of different adaptation to effort, recovery capacity/velocity, and energy metabolism.

Conclusions

Based on the findings here collected, the majority of evidence supports effectiveness of WBC in relieving symptomatology of the whole set of inflammatory conditions that could affect an athlete. A small number of studies that did not report any positive effects should, however, not be neglected. The same applies to improvement of post-exercise recovery, and noteworthy, to limiting or even preventing EIMD. The perception of WBC is changing from a conventionally intended symptomatic therapy to a stimulating treatment able to enhance the anti-inflammatory and -oxidant barriers and to counteract harmful stimuli. Importantly, cooling effectiveness depends on the percentage of fat mass of a subject and the starting fitness level. These results, combined with evidence that WBC somehow mimics exercise, at least in its ability to induce a pulsatile expression of myokines (IL-6, irisin), open another window of possible therapeutic strategies for obesity and type 2 diabetes.

As above highlighted, some of the applied WBC protocols have been ineffective in inducing appreciable modifications of certain biochemical parameters. However, in these cases, the final clinical output (in a subjective assessment: in terms of pain, soreness, stress, and recovery) was significantly improved even when compared to other recovery strategies.

WBC, used either as a therapy or stimulation, is a medical treatment and as such it has contraindications and standard safety procedures. The undeniable risks for the users can be rendered negligible if all the procedures are conducted following precise rules under supervision of highly-skilled personnel. If these procedures are carefully followed, WBC is absolutely safe.

The scientific debate on WBC, often shaded by non-scientific discussions hold in newspapers and web dictated by curiosity or accidents (recent incident in a non-controlled cryocabin), needs consensus and international cooperation for building up wide and controlled studies.

Author contributions

GL and EZ: conception and design, data acquisition; drafting paper; final approval; agreement for all the aspects of the work. GB: conception and design, data acquisition; critical revision; final approval; agreement for all the aspects of the work.

This work has been funded by an unrestricted grant from the Italian Ministry of Health and grant from the Polish Ministry of Science and Higher Education No. 0026/RS3/2015/53.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

  • Banfi G., Krajewska M., Melegati G., Patacchini M. (2008). Effects of whole-body cryotherapy on haematological values in athletes . Br. J. Sports Med. 42 , 858. [ PubMed ] [ Google Scholar ]
  • Banfi G., Lombardi G., Colombini A., Lippi G. (2010a). Bone metabolism markers in sports medicine . Sports Med. 40 , 697–714. 10.2165/11533090-000000000-00000 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Banfi G., Lombardi G., Colombini A., Melegati G. (2010b). Whole-body cryotherapy in athletes . Sports Med. 40 , 509–517. 10.2165/11531940-000000000-00000 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Banfi G., Melegati G., Barassi A., d'Eril G. M. (2009a). Effects of the whole-body cryotherapy on NTproBNP, hsCRP and troponin I in athletes . J. Sci. Med. Sport 12 , 609–610. 10.1016/j.jsams.2008.06.004 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Banfi G., Melegati G., Barassi A., Dogliotti G., d'Eril G. M., Dugue B., et al. (2009b). Effects of whole-body cryotherapy on serum mediators of inflammation and serum muscle enzymes in athletes . J. Therm. Biol. 34 , 55–59. 10.1016/j.jtherbio.2008.10.003 [ CrossRef ] [ Google Scholar ]
  • Bettoni L., Bonomi F. G., Zani V., Manisco L., Indelicato A., Lanteri P., et al.. (2013). Effects of 15 consecutive cryotherapy sessions on the clinical output of fibromyalgic patients . Clin. Rheumatol. 32 , 1337–1345. 10.1007/s10067-013-2280-9 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Bleakley C. M., Hopkins T. J. (2010). Is it possible to achieve optimal levels of tissue cooling in cryotherapy? Phys. Ther. Rev. 15 , 344–351. 10.1179/174328810X12786297204873 [ CrossRef ] [ Google Scholar ]
  • Bleakley C. M., Bieuzen F., Davison G. W., Costello J. T. (2014). Whole-body cryotherapy: empirical evidence and theoretical perspectives . Open Access J. Sports Med. 5 , 25–36. 10.2147/OAJSM.S41655 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Boström P., Wu J., Jedrychowski M. P., Korde A., Ye L., Lo J. C., et al.. (2012). A PGC1-α-dependent myokine that drives brown-fat-like development of white fat and thermogenesis . Nature 481 , 463–468. 10.1038/nature10777 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Bouzigon R., Grappe F., Ravier G., Dugue B. (2016). Whole-body and partial-body cyostimulation/cryotherapy: current technologies and practical applications . J. Therm. Biol. 61 , 67–81. 10.1016/j.jtherbio.2016.08.009 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Cholewka A., Stanek A., Sieron A., Drzazga Z. (2012). Thermography study of skin response due to whole-body cryotherapy . Skin Res. Technol. 18 , 180–187. 10.1111/j.1600-0846.2011.00550.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Collins N. C. (2008). Is ice right? Does cryotherapy improve outcome for acute soft tissue injury? Emer. Med. J. 25 , 65–68. 10.1136/emj.2007.051664 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Costello J. T., Algar L. A., Donnelly A. E. (2012a). Effects of whole-body cryotherapy (−110°C) on proprioception and indices of muscle damage . Scand. J. Med. Sci. Sports 22 , 190–198. 10.1111/j.1600-0838.2011.01292.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Costello J. T., Baker P. R., Minett G. M., Bieuzen F., Stewart I. B., Bleakley C. (2015). Whole-body cryotherapy (extreme cold air exposure) for preventing and treating muscle soreness after exercise in adults . Cochrane Database Syst. Rev. CD010789. 10.1002/14651858.CD010789.pub2 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Costello J. T., Culligan K., Selfe J., Donnelly A. E. (2012b). Muscle, skin and core temperature after −110°C cold air and 8°C water treatment . PLoS ONE 7 :e48190. 10.1371/journal.pone.0048190 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Costello J. T., Donnelly A. E., Karki A., Selfe J. (2014). Effects of whole body cryotherapy and cold water immersion on knee skin temperature . Int. J. Sports Med. 35 , 35–40. 10.1055/s-0033-1343410 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Dugué B. M. (2015). An attempt to improve Ferreira-Junior model concerning the anti-inflammatory action of whole-body cryotherapy after exercise induced muscular damage (EIMD) . Front. Physiol. 6 :35. 10.3389/fphys.2015.00035 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Dulian K., Laskowski R., Grzywacz T., Kujach S., Flis D. J., Smaruj M., et al.. (2015). The whole body cryostimulation modifies irisin concentration and reduces inflammation in middle aged, obese men . Cryobiology 71 , 398–404. 10.1016/j.cryobiol.2015.10.143 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Ferreira-Junior J. B., Bottaro M., Loenneke J. P., Vieira A., Vieira C. A., Bemben M. G. (2014). Could whole-body cryotherapy (below −100°C) improve muscle recovery from muscle damage? Front. Physiol. 5 :247. 10.3389/fphys.2014.00247 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Furmanek M. P., Slomka K., Juras G. (2014). The effects of cryotherapy on proprioception system . Biomed. Res. Int. 2014 :696397. 10.1155/2014/696397 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Galliera E., Dogliotti G., Melegati G., Corsi Romanelli M. M., Cabitza P., Banfi G. (2012). Bone remodelling biomarkers after whole body cryotherapy (WBC) in elite rugby players . Injury 44 , 1117–1121. 10.1016/j.injury.2012.08.057 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Grasso D., Lanteri P., Di Bernardo C., Mauri C., Porcelli S., Colombini A., et al.. (2014). Salivary steroid hormones response to whole-body cryotherapy in elite rugby players . J. Biol. Regul. Homeost. Agents 28 , 291–300. [ PubMed ] [ Google Scholar ]
  • Hammond L. E., Cuttell S., Nunley P., Meyler J. (2014). Anthropometric characteristics and sex influence magnitude of skin cooling following exposure to whole body cryotherapy . Biomed Res. Int. 2014 :628724. 10.1155/2014/628724 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Hausswirth C., Louis J., Bieuzen F., Pournot H., Fournier J., Filliard J. R., et al.. (2011). Effects of whole-body cryotherapy vs. far-infrared vs. passive modalities on recovery from exercise-induced muscle damage in highly-trained runners . PLoS ONE 6 :e27749. 10.1371/journal.pone.0027749 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Hausswirth C., Schaal K., Le Meur Y., Bieuzen F., Filliard J. R., Volondat M., et al.. (2013). Parasympathetic activity and blood catecholamine responses following a single partial-body cryostimulation and a whole-body cryostimulation . PLoS ONE 8 :e72658. 10.1371/journal.pone.0072658 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Hornery D. J., Papalia S., Mujika I., Hahn A. (2005). Physiological and performance benefits of halftime cooling . J. Sci. Med. Sport 8 , 15–25. 10.1016/S1440-2440(05)80020-9 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Jastrzabek R., Straburzynska-Lupa A., Rutkowski R., Romanowski W. (2013). Effects of different local cryotherapies on systemic levels of TNF-α, IL-6, and clinical parameters in active rheumatoid arthritis . Rheumatol. Int. 33 , 2053–2060. 10.1007/s00296-013-2692-5 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Krüger M., de Marees M., Dittmar K. H., Sperlich B., Mester J. (2015). Whole-body cryotherapy's enhancement of acute recovery of running performance in well-trained athletes . Int. J. Sports Physiol. Perform. 10 , 605–612. 10.1123/ijspp.2014-0392 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lee P., Linderman J. D., Smith S., Brychta R. J., Wang J., Idelson C., et al.. (2014). Irisin and FGF21 are cold-induced endocrine activators of brown fat function in humans . Cell Metab. 19 , 302–309. 10.1016/j.cmet.2013.12.017 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lombardi G., Colombini A., Freschi M., Tavana R., Banfi G. (2011). Seasonal variation of bone turnover markers in top-level female skiers . Eur. J. Appl. Physiol. 111 , 433–440. 10.1007/s00421-010-1664-7 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lombardi G., Colombini A., Porcelli S., Mauri C., Zani V., Bonomi F. G., et al. (2014). Muscular damage and kidney function in rugby players after daily whole body cryostimulation . Physiol. J. 2014 :790540 10.1155/2014/790540 [ CrossRef ] [ Google Scholar ]
  • Lombardi G., Lanteri P., Graziani G., Colombini A., Banfi G., Corsetti R. (2012). Bone and energy metabolism parameters in professional cyclists during the Giro d'Italia 3-weeks stage race . PLoS ONE 7 :e42077. 10.1371/journal.pone.0042077 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lombardi G., Lanteri P., Porcelli S., Mauri C., Colombini A., Grasso D., et al.. (2013a). Hematological profile and martial status in rugby players during whole body cryostimulation . PLoS ONE 8 :e55803. 10.1371/journal.pone.0055803 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lombardi G., Lippi G., Banfi G. (2013b). Iron requirements and iron status of athletes , in Sports Nutrition , ed Moughan R. J.(Hoboken, NJ: Wiley-Blackwell; John Wiley & Sons, Inc.), 229–241. 10.1002/9781118692318.ch19 [ CrossRef ] [ Google Scholar ]
  • Lombardi G., Sanchis-Gomar F., Perego S., Sansoni V., Banfi G. (2016). Implications of exercise-induced adipo-myokines in bone metabolism . Endocrine 54 , 284–305. 10.1007/s12020-015-0834-0 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lubkowska A., Szygula Z. (2010). Changes in blood pressure with compensatory heart rate decrease and in the level of aerobic capacity in response to repeated whole-body cryostimulation in normotensive, young and physically active men . Int. J. Occup. Med. Environ. Health 23 , 367–375. 10.2478/v10001-010-0037-0 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lubkowska A., Banfi G., Dolegowska B., d'Eril G. V., Luczak J., Barassi A. (2010a). Changes in lipid profile in response to three different protocols of whole-body cryostimulation treatments . Cryobiology 61 , 22–26. 10.1016/j.cryobiol.2010.03.010 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lubkowska A., Dolegowska B., Szygula Z. (2012). Whole-body cryostimulation - potential beneficial treatment for improving antioxidant capacity in healthy men - significance of the number of sessions . PLoS ONE 7 :e46352. 10.1371/journal.pone.0046352 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lubkowska A., Dudzinska W., Bryczkowska I., Dolegowska B. (2015). Body composition, lipid profile, adipokine concentration, and antioxidant capacity changes during interventions to treat overweight with exercise programme and whole-body cryostimulation . Oxid. Med. Cell. Longev. 2015 :803197. 10.1155/2015/803197 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lubkowska A., Szygula Z., Chlubek D., Banfi G. (2011). The effect of prolonged whole-body cryostimulation treatment with different amounts of sessions on chosen pro- and anti-inflammatory cytokines levels in healthy men . Scand. J. Clin. Lab. Invest. 71 , 419–425. 10.3109/00365513.2011.580859 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Lubkowska A., Szygula Z., Klimek A. J., Torii M. (2010b). Do sessions of cryostimulation have influence on white blood cell count, level of IL6 and total oxidative and antioxidative status in healthy men? Eur. J. Appl. Physiol. 109 , 67–72. 10.1007/s00421-009-1207-2 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Mila-Kierzenkowska C., Jurecka A., Wozniak A., Szpinda M., Augustynska B., Wozniak B. (2013). The effect of submaximal exercise preceded by single whole-body cryotherapy on the markers of oxidative stress and inflammation in blood of volleyball players . Oxid. Med. Cell. Longev. 2013 :409567. 10.1155/2013/409567 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Miller E., Markiewicz L., Saluk J., Majsterek I. (2012). Effect of short-term cryostimulation on antioxidative status and its clinical applications in humans . Eur. J. Appl. Physiol. 112 , 1645–1652. 10.1007/s00421-011-2122-x [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Peake J. M., Della Gatta P., Suzuki K., Nieman D. C. (2015). Cytokine expression and secretion by skeletal muscle cells: regulatory mechanisms and exercise effects . Exerc. Immunol. Rev. 21 , 8–25. [ PubMed ] [ Google Scholar ]
  • Poppendieck W., Faude O., Wegmann M., Meyer T. (2013). Cooling and performance recovery of trained athletes: a meta-analytical review . Int. J. Sports Physiol. Perform. 8 , 227–242. 10.1123/ijspp.8.3.227 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Pournot H., Bieuzen F., Louis J., Mounier R., Fillard J. R., Barbiche E., et al.. (2011). Time-course of changes in inflammatory response after whole-body cryotherapy multi exposures following severe exercise . PLoS ONE 6 :e22748. 10.1371/annotation/0adb3312-7d2b-459c-97f7-a09cfecf5881 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Roberts L. A., Raastad T., Markworth J. F., Figueiredo V. C., Egner I. M., Shield A., et al.. (2015). Post-exercise cold water immersion attenuates acute anabolic signalling and long-term adaptations in muscle to strength training . J. Physiol. 593 , 4285–4301. 10.1113/JP270570 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Robinson N., Sottas P. E., Pottgiesser T., Schumacher Y. O., Saugy M. (2011). Stability and robustness of blood variables in an antidoping context . Int. J. Lab. Hematol . 33 , 146–153. 10.1111/j.1751-553X.2010.01256.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Russell M., Birch J., Love T., Cook C. J., Bracken R. M., Taylor T., et al.. (2017). The effects of a single whole-body cryotherapy exposure on physiological, performance, and perceptual responses of professional academy soccer players after repeated sprint exercise . J. Strength Cond. Res. 31 , 415–421. 10.1519/JSC.0000000000001505 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Sansoni V., Vernillo G., Perego S., Barbuti A., Merati G., Schena F., et al.. (2017). Bone turnover response is linked to both acute and established metabolic changes in ultra-marathon runners . Endocrine . 56 , 196–204. 10.1007/s12020-016-1012-8 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Schaal K., LE Meur Y., Louis J., Filliard J. R., Hellard P., Casazza G., et al.. (2015). Whole-body cryostimulation limits overreaching in elite synchronized swimmers . Med. Sci. Sports Exerc. 47 , 1416–1425. 10.1249/MSS.0000000000000546 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Selfe J., Alexander J., Costello J. T., May K., Garratt N., Atkins S., et al.. (2014). The effect of three different (−135°C) whole body cryotherapy exposure durations on elite rugby league players . PLoS ONE 9 :e86420. 10.1371/journal.pone.0086420 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Slattery K., Bentley D., Coutts A. J. (2015). The role of oxidative, inflammatory and neuroendocrinological systems during exercise stress in athletes: implications of antioxidant supplementation on physiological adaptation during intensified physical training . Sports Med. 45 , 453–471. 10.1007/s40279-014-0282-7 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Smolander J., Westerlund T., Uusitalo A., Dugue B., Oksa J., Mikkelsson M. (2006). Lung function after acute and repeated exposures to extremely cold air (−110°C) during whole-body cryotherapy . Clin. Physiol. Funct. Imaging 26 , 232–234. 10.1111/j.1475-097X.2006.00675.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Sottas P. E., Robinson N., Saugy M. (2010). The athlete's biological passport and indirect markers of blood doping . Handb. Exp. Pharmacol. 195 , 305–326. 10.1007/978-3-540-79088-4_14 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Straburzyńska-Lupa A., Konarska A., Nowak A., Straburzyńska-Migaj E., Konarski J., Kijewski K., et al. (2007). Effect of whole-body cryotherapy on selected blood chemistry parameters in professional field hockey players . Fizjoter. Pol. 7 , 15–20. [ Google Scholar ]
  • Sutkowy P., Augustynska B., Wozniak A., Rakowski A. (2014). Physical exercise combined with whole-body cryotherapy in evaluating the level of lipid peroxidation products and other oxidant stress indicators in kayakers . Oxid. Med. Cell. Longev. 2014 :402631. 10.1155/2014/402631 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Szygula Z., Lubkowska A., Giemza C., Skrzek A., Bryczkowska I., Dolegowska B. (2014). Hematological parameters, and hematopoietic growth factors: EPO and IL-3 in response to whole-body cryostimulation (WBC) in military academy students . PLoS ONE 9 :e93096. 10.1371/journal.pone.0093096 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Wozniak A., Mila-Kierzenkowska C., Szpinda M., Chwalbinska-Moneta J., Augustynska B., Jurecka A. (2013). Whole-body cryostimulation and oxidative stress in rowers: the preliminary results . Arch. Med. Sci. 9 , 303–308. 10.5114/aoms.2012.30835 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Wozniak A., Wozniak B., Drewa G., Mila-Kierzenkowska C., Rakowski A. (2007). The effect of whole-body cryostimulation on lysosomal enzyme activity in kayakers during training . Eur. J. Appl. Physiol. 100 , 137–142. 10.1007/s00421-007-0404-0 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Yamauchi T., Kim S., Nogami S., Kawano A. D. (1981a). Extreme cold treatment (−150°C) on the whole body in rheumatoid arthritis . Rev. Rheum 48 ( Suppl. ):P1054. [ Google Scholar ]
  • Yamauchi T., Nogami S., Miura K. (1981b). Various application of the extreme cryotherapy and strenuous exercise program . Physiother. Rehab. 5 , 35–39. [ Google Scholar ]
  • Zalewski P., Bitner A., Slomko J., Szrajda J., Klawe J. J., Tafil-Klawe M., et al.. (2014). Whole-body cryostimulation increases parasympathetic outflow and decreases core body temperature . J. Therm. Biol. 45 , 75–80. 10.1016/j.jtherbio.2014.08.001 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Ziemann E., Olek R. A., Grzywacz T., Antosiewicz J., Kujach S., Luszczyk M., et al.. (2013). Whole-body cryostimulation as an effective method of reducing low-grade inflammation in obese men . J. Physiol. Sci. 63 , 333–343. 10.1007/s12576-013-0269-4 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Ziemann E., Olek R. A., Grzywacz T., Kaczor J. J., Antosiewicz J., Skrobot W., et al.. (2014). Whole-body cryostimulation as an effective way of reducing exercise-induced inflammation and blood cholesterol in young men . Eur. Cytokine Netw. 25 , 14–23. 10.1684/ecn.2014.0349 [ PubMed ] [ CrossRef ] [ Google Scholar ]
  • Ziemann E., Olek R. A., Kujach S., Grzywacz T., Antosiewicz J., Garsztka T., et al.. (2012). Five-day whole-body cryostimulation, blood inflammatory markers, and performance in high-ranking professional tennis players . J. Athl. Train. 47 , 664–672. 10.4085/1062-6050-47.6.13 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]

Duke University Libraries

Literature Reviews

  • Types of reviews
  • Getting started

Types of reviews and examples

Choosing a review type.

  • 1. Define your research question
  • 2. Plan your search
  • 3. Search the literature
  • 4. Organize your results
  • 5. Synthesize your findings
  • 6. Write the review
  • Thompson Writing Studio This link opens in a new window
  • Need to write a systematic review? This link opens in a new window

an updated review of the literature

Contact a Librarian

Ask a Librarian

Overview of types of literature reviews

Made with  Visme Infographic Maker

  • Literature (narrative)
  • Scoping / Evidence map
  • Meta-analysis

Characteristics:

  • Provides examination of recent or current literature on a wide range of subjects
  • Varying levels of completeness / comprehensiveness, non-standardized methodology
  • May or may not include comprehensive searching, quality assessment or critical appraisal

Mitchell, L. E., & Zajchowski, C. A. (2022). The history of air quality in Utah: A narrative review.  Sustainability ,  14 (15), 9653.  doi.org/10.3390/su14159653

  • Assessment of what is already known about an issue
  • Similar to a systematic review but within a time-constrained setting
  • Typically employs methodological shortcuts, increasing risk of introducing bias, includes basic level of quality assessment
  • Best suited for issues needing quick decisions and solutions (i.e., policy recommendations)

Learn more about the method:

Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012). Evidence summaries: the evolution of a rapid review approach.  Systematic reviews, 1 (1), 1-9.  https://doi.org/10.1186/2046-4053-1-10

Virginia Commonwealth University Libraries. (2021). Rapid Review Protocol .

Quarmby, S., Santos, G., & Mathias, M. (2019). Air quality strategies and technologies: A rapid review of the international evidence.  Sustainability, 11 (10), 2757.  https://doi.org/10.3390/su11102757

  • Compiles evidence from multiple reviews into one document
  • Often defines a broader question than is typical of a traditional systematic review.

Choi, G. J., & Kang, H. (2022). The umbrella review: a useful strategy in the rain of evidence.  The Korean Journal of Pain ,  35 (2), 127–128.  https://doi.org/10.3344/kjp.2022.35.2.127

Aromataris, E., Fernandez, R., Godfrey, C. M., Holly, C., Khalil, H., & Tungpunkom, P. (2015). Summarizing systematic reviews: Methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare , 13(3), 132–140. https://doi.org/10.1097/XEB.0000000000000055

Rojas-Rueda, D., Morales-Zamora, E., Alsufyani, W. A., Herbst, C. H., Al Balawi, S. M., Alsukait, R., & Alomran, M. (2021). Environmental risk factors and health: An umbrella review of meta-analyses.  International Journal of Environmental Research and Public Dealth ,  18 (2), 704.  https://doi.org/10.3390/ijerph18020704

  • Main purpose is to map out and categorize existing literature, identify gaps in literature
  • Search comprehensiveness determined by time/scope constraints, could take longer than a systematic review
  • No formal quality assessment or critical appraisal

Learn more about the methods :

Arksey, H., & O'Malley, L. (2005) Scoping studies: towards a methodological framework.  International Journal of Social Research Methodology ,  8 (1), 19-32.  https://doi.org/10.1080/1364557032000119616

Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science: IS, 5, 69. https://doi.org/10.1186/1748-5908-5-69

Miake-Lye, I. M., Hempel, S., Shanman, R., & Shekelle, P. G. (2016). What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products.  Systematic reviews, 5 (1), 1-21.  https://doi.org/10.1186/s13643-016-0204-x

Example : 

Rahman, A., Sarkar, A., Yadav, O. P., Achari, G., & Slobodnik, J. (2021). Potential human health risks due to environmental exposure to nano-and microplastics and knowledge gaps: A scoping review.  Science of the Total Environment, 757 , 143872.  https://doi.org/10.1016/j.scitotenv.2020.143872

  • Seeks to systematically search for, appraise, and synthesize research evidence
  • Adheres to strict guidelines, protocols, and frameworks
  • Time-intensive and often take months to a year or more to complete. 
  • The most commonly referred to type of evidence synthesis. Sometimes confused as a blanket term for other types of reviews.

Gascon, M., Triguero-Mas, M., Martínez, D., Dadvand, P., Forns, J., Plasència, A., & Nieuwenhuijsen, M. J. (2015). Mental health benefits of long-term exposure to residential green and blue spaces: a systematic review.  International Journal of Environmental Research and Public Health ,  12 (4), 4354–4379.  https://doi.org/10.3390/ijerph120404354

  • Statistical technique for combining results of quantitative studies to provide more precise effect of results
  • Aims for exhaustive, comprehensive searching
  • Quality assessment may determine inclusion/exclusion criteria
  • May be conducted independently or as part of a systematic review

Berman, N. G., & Parker, R. A. (2002). Meta-analysis: Neither quick nor easy. BMC Medical Research Methodology , 2(1), 10. https://doi.org/10.1186/1471-2288-2-10

Hites R. A. (2004). Polybrominated diphenyl ethers in the environment and in people: a meta-analysis of concentrations.  Environmental Science & Technology ,  38 (4), 945–956.  https://doi.org/10.1021/es035082g

Flowchart of review types

  • Review Decision Tree - Cornell University For more information, check out Cornell's review methodology decision tree.
  • LitR-Ex.com - Eight literature review methodologies Learn more about 8 different review types (incl. Systematic Reviews and Scoping Reviews) with practical tips about strengths and weaknesses of different methods.
  • << Previous: Getting started
  • Next: 1. Define your research question >>
  • Last Updated: Feb 15, 2024 1:45 PM
  • URL: https://guides.library.duke.edu/lit-reviews

Duke University Libraries

Services for...

  • Faculty & Instructors
  • Graduate Students
  • Undergraduate Students
  • International Students
  • Patrons with Disabilities

Twitter

  • Harmful Language Statement
  • Re-use & Attribution / Privacy
  • Support the Libraries

Creative Commons License

  • Open access
  • Published: 22 February 2024

A scoping review on the methodological and reporting quality of scoping reviews in China

  • Xinyu Xue 2 ,
  • Xintong Tang 2 ,
  • Shanshan Liu 2 ,
  • Ting Yu 3 ,
  • Zhonglan Chen 3 ,
  • Ningsu Chen 2 &
  • Jiajie Yu 1 , 2  

BMC Medical Research Methodology volume  24 , Article number:  45 ( 2024 ) Cite this article

10 Accesses

1 Altmetric

Metrics details

Scoping reviews have emerged as a valuable method for synthesizing emerging evidence, providing a comprehensive contextual overview, and influencing policy and practice developments. The objective of this study is to provide an overview of scoping reviews conducted in Chinese academic institutions over the last decades.

We conducted a comprehensive search of nine databases and six grey literature databases for scoping reviews conducted in Chinese academic institutions. The reporting quality of the included reviews was assessed using the Preferred Reporting Items for PRISMA-ScR checklist. We performed both quantitative and qualitative analyses, examining the conduct of the scoping reviews and exploring the breadth of research topics covered. We used Chi-squared and Wilcoxon rank-sum tests to compare methodological issues and reporting quality in English and Chinese-language reviews.

A total of 392 reviews published between 2013 and 2022 were included, 238 English-reported reviews and 154 Chinese-reported reviews, respectively. The primary purposes of these reviews were to map and summarize the evidence, with a particular focus on health and nursing topics. 98.7% of reviews explicitly used the term “scoping review”, and the Arksey and O’Malley framework was the most frequently cited framework. Thirty-five English-reported scoping reviews provided a protocol for scoping review. PubMed was the most common source in English-reported reviews and CNKI in Chinese-reported reviews. Reviews published in English were more likely to search the grey literature ( P  = 0.005), consult information specialists ( P  < 0.001) and conduct an updated search ( P  = 0.012) than those in Chinese. Reviews published in English had a significantly high score compared to those published in Chinese (16 vs. 14; P  < 0.001). The reporting rates in English-reported reviews were higher than those in Chinese reviews for seven items, but lower for structured summary ( P  < 0.001), eligibility criteria ( P  < 0.001), data charting process ( P  = 0.009) and data items ( P  = 0.015).

There has been a significant increase in the number of scoping reviews conducted in Chinese academic institutions each year since 2020. While the research topics covered are diverse, the overall reporting quality of these reviews is need to be improved. And there is a need for greater standardization in the conduct of scoping reviews in Chinese academic institutions.

Peer Review reports

Introduction

Along with the increased production of primary research, the conduct and publication of evidence synthesis have also increased over time [ 1 ]. To address various questions for policymakers and other stakeholders, different types of reviews have emerged [ 2 ], with 48 types of reviews identified by 2019 [ 3 ]. One of the review types is the “scoping review”, also known as a “scoping study”, or “mapping review” [ 4 ]. Scoping reviews are a valuable approach to synthesizing emerging evidence, providing a comprehensive overview of the context and the potential to influence policy and practice developments [ 5 , 6 ]. Scoping reviews have been widely conducted in various fields, including health, technology and social sciences over the past decades [ 7 ].

Scoping reviews follow similar processes to systematic reviews in terms of identifying and analyzing relevant literature on a specific topic [ 8 ]. Scoping review characteristically involves the development, assimilation, and synthesis of a broad base of evidence derived from a diverse range of published research and grey literature research [ 9 ]. It aims to clarify the key concepts and characteristics that underpin a research area, determine a precise volume of literature and studies available, or can be a precursor to a systematic review [ 10 ]. Unlike systematic reviews, scoping reviews do not require a quality assessment of individual studies or the integration of evidence from different studies [ 11 ]. While a scoping study requires a framework to investigate existing literature, it does not involve assessing the weight of evidence for particular interventions or policies [ 12 ].

Scoping review is widely used to answer board research questions now. The concept was initially proposed by Mays in 2001 [ 13 ] and later used by Arksey and O’Malley, who provided the first guidance on conducting scoping review in 2005 [ 14 ]. In 2014, the JBI and JBI collaboration published their guidance on scoping review [ 15 ] and updated it in 2020 [ 16 , 17 ]. As a result, the publication of scoping reviews has significantly increased. To improve the methodological and reporting quality of scoping reviews, the Preferred Reporting Items for Systematic Reviews extension for Scoping Review [ 18 ] (PRISMA-ScR) was published in 2018.

The research community involved in scoping reviews, mainly from Canada, the United States, the United Kingdom, and Australia, shows a steady increase in literature and maintains a relatively high growth rate [ 4 ]. China is currently in the early stages of introducing and familiarizing itself with the scoping review methodology, with few documented examples of its practical application in the field of medicine [ 19 ]. Additionally, some review articles in China, although conceptually similar to scoping reviews, did not follow the standardized methodology and therefore could not be classified as scoping reviews [ 20 , 21 ]. This may be attributed to a lack of awareness and a delay in the adoption of scoping reviews by Chinese academic authors.

In light of the above situation and the growing utilization of scoping reviews in China, our study aims to (1) examine volume, scope and distribution of scoping reviews conducted in Chinese academic institutions; (2) summarize the purpose, topics, and methodological issues in these scoping reviews; (3) explore the extent to which scoping review adheres to reporting guidelines.

Methods and analysis

We used Arksey and O’Malley’s framework [ 14 ] on scoping review to guide our study. This protocol was registered on the open science framework ( https://osf.io/f9u6q/ ).

Identify the research questions

Our research questions were as follows.

1) What is the volume, scope and distribution of scoping reviews conducted in Chinese academic institutions?

2) What are the purposes, topics and methodological issues in the included scoping reviews?

3) To what extent do these scoping reviews adhere to the PRISMA ScR reporting guidelines?

Eligibility criteria

We included all scoping reviews that met the following criteria: (1) utilized a scoping review of the literature approach with a description of the synthesis method used and (2) focused on the field of health/medicine. We excluded studies that (1) did not synthesize literature, such as complete scoping of surveillance or administrative databases; (2) primarily described scoping review methods or guidelines and (3) full text was not available. We defined scoping reviews conducted in Chinese academic institutions as the corresponding author’s affiliations located in mainland China, Hong Kong, Macau or Taiwan.

Identify relevant studies

We conducted a comprehensive literature search from the inception of the following five English electronic databases and four Chinese databases until Dec 2021, with an update to Dec 2022: PubMed, EMBASE, EBSCO, Web of Science, The Cochrane Library, SinoMed, VIP Chinese periodical Service, Wanfang Data Knowledge Service, and China National Knowledge Infrastructure (CNKI). Grey literature (e.g. thesis and dissertation, newspaper, conference paper) and some web search engines (e.g. Google, Google Scholar, Baidu, Baidu Scholar) were also searched. The search strategy was not restricted by study design and an expert information specialist collaborated with the research team. The search strategies were listed in Appendix 1 We also cross-checked the included studies and references of a relevant scoping review.

Study selection

Inter-rater agreement for study inclusion was calculated using percent agreement. If the agreement exceed 75% among the team members, we proceeded to the next stage. All title and abstract screening and full-text screening were performed independently by at least two review authors (Xue XY, Tang XT, Liu SS, Yu T) using a pre-defined form. Any discrepancies were resolved through consensus or involving a third reviewer (Yu JJ) when necessary.

Charting the data

The following general information was collected from each eligible study: published year, regions, affiliations, journal name, study population, number of studies included, and funding source. We also collected details on the purpose of the scoping review (e.g., identified evidence gaps, future research opportunities, implications for policy or practice), as well as the topic addressed. We collected methodological information including the definition of “scoping review”, utilization of established methodological guidance (e.g., Arksey and O’Malley, Levac, Joanna Briggs Institute, or others), protocol and registration, research question, inclusion criteria, eligible study design (e.g., primary studies, secondary studies, both, or others), search strategy, databases searched, additional search resources (e.g. explore breadth/extent of evidence, grey literature, consulted experts, crosscheck references), title and abstract screening, full-text screening, pre-defined charting form, flow diagram, result presentation (tables and/or diagrams), the implication for research and practice [ 22 ].

We assessed the reporting quality of eligible scoping reviews using the PRISMA-ScR checklist [ 18 ], which includes 20 items and 2 optional items for critical appraisal of individual studies. Each item was determined with the option of “yes” or “no”, allocating 1 point if the study met the requirement for a specific item and 0 if not. A total score ranging from 0 to 22 was developed.

Data extraction and reporting quality from each eligible study were conducted by four reviewers, a pilot study was performed before formal extraction, and the interrater agreement percentage needed to be > 75%. Discrepancies were resolved by consensus or the involvement of a third reviewer.

Collating and summarizing results

We conducted both quantitative and qualitative analyses on the scoping reviewsThe quantitative analysis involved examining the distribution of reviews, methodological issues, and reporting quality. For quantitative analysis, frequencies and proportions were calculated for the categorical variables and mean (SD), median (range) or median (IQR) were used to analyze the continuous variables. Word clouds were generated using the online program WordClouds to visualize the synthesis topics (Zygomatic, 2022) ( https://www.wordclouds.com ).

We performed a comparative analysis of methodological issues and reporting quality between eligible scoping reviews published in English and Chinese. Either X 2 or Fisher’s exact tests were used for the analysis of categorical variables and the Wilcoxon rank-sum test was used for continuous data with a non-normal distribution. For qualitative analysis, two reviewers independently categorized the key components, and the results were subsequently discussed by the research team.

Search and selection of scoping review

A total of 2958 citations relevant to scoping review were searched, and 2046 studies were included for screening after duplication. After reading 589 potentially relevant full-text papers, 392 articles were finally included (Fig.  1 ). The interrater agreements among the four reviewers were good, with agreement rates of 92.2% at the title and abstract screening, 94.8% at the full-text reading, and 95.3% at the table extraction.

figure 1

Details the flow of information through the different phases of the review; map out the number of records identified, included and excluded, and the reasons for their exclusion

Study characteristics of including scoping reviews

The scoping reviews included in our study were published between 2013 and 2022, with a significant majority ( n  = 340, 86.7%) published after 2020 (Fig.  2 ). The median number of studies included in these scoping reviews was 29 (Range: 5-6430). Among the included reviews, 238 (60.7%) were reported in English. A total of 217 journals were involved in publishing the scoping review, with the Chinese Journal of Nursing ( n  = 14, 3.6%) and the International Journal of Environmental Research and Public Health ( n  = 10, 2.6%) being the most commonly published Chinese and English journals, respectively (Table  1 ). Most of the reviews were conducted in Beijing ( n  = 66, 16.8%), Hong Kong ( n  = 59, 15.1%), and Shanghai ( n  = 44, 11.2%) (Appendix 2 ). The reviews originated from 166 institutions, primarily universities and hospitals (Table  1 ). 274 (69.9%) received funding support, with majority (98.5%) being publicly sponsored. Notably, the distribution of these characteristics was different between Chinese reviews and English reviews (Table  1 ).

figure 2

Published year

The annual number of scoping reviews conducted by Chinese academic institutions is visually represented in the form of a line graph

The study population for the included scoping review primarily consisted of patients ( n  = 200, 51.0%) and healthcare professionals ( n  = 60, 15.3%) (Appendix 3 ). The main purposes of the reviews were to map and summarize existing evidence ( n  = 230, 58.7%), followed by to identify and/or address knowledge gaps ( n  = 111, 28.3%) (Appendix 4 ). The topics covered in the reviews were diverse, with health and nursing being the common topics (Fig.  3 ).

figure 3

Word cloud of topics

The most common topic in the 392 scoping reviews is displayed, with the size of the topics in the word cloud corresponding to the frequency of their show

Method characteristics of including scoping reviews

Of the 392 scoping reviews, the majority (387, 98.7%) explicitly identified themselves as “scoping review”, while 122 (31.1%) provided a definition of the term ‘scoping review’. The most frequently cited framework for conducting scoping review was Arksey and O’Malley framework (2005), referenced in 157 reviews (40.1%). The research question was clearly stated in 39.8% of the reviews, while 75.8% clearly reported their inclusion criteria.

Thirty-four (8.7%) English-reported scoping reviews provided a protocol of scoping review, and 20 (5.1%) were registered on the Open Science Framework (OSF). The majority of reviews (369, 94.1%) searched more than one database, with PubMed ( n  = 184, 77.3%) being the most common source for English-language reviews and CNKI ( n  = 137, 89.0%) for Chinese-language reviews. In terms of additional search, English-reported reviews were more likely to search the grey literature (70 vs. 39, P  = 0.005), consult information specialists (18 vs. 3, P  < 0.001) and conducted an updated search (16 vs. 2, P  = 0.012) compared to Chinese-reported reviews (Table  2 ).

Approximately 80% of reviews screened title/abstract and full-text articles with more than two reviewers. A predefined abstraction form was used in 30.4% of the reviews and data extraction involved more than two reviewers in 57.7% of reviews. Among the included reviews, 56 reviews (14.3%) assessed the quality of the studies and six reviews (1.5%) conducted a meta-analyses. More than 75% of included reviews provided a study flow chart, while the difference was significant between reviews in English language and Chinese (212 vs. 90, P  < 0.001). Additionally, nearly 90% of the reviews presented their results in tabular form, while 31.1% used graphical representation. In terms of discussion, a higher proportion of scoping reviews published in English journals compared to Chinese journals mentioned the limitations of their studies (193 vs. 50, P  < 0.001) as well as their strengths (68 vs. 22, P  = 0.001) (Table  2 ).

Reporting quality of including scoping reviews

Of the 22 items, six (27.3%) were adequately reported: identification of the report as a scoping review in the title (93.1%), statement of eligibility criteria for included evidence (90.6%), description of all sources of information used in the search (100.0%), process for synthesizing the results (88.3%), methods of summarizing the evidence (100.0%) and presentation of a conclusion (97.4%). On the other hand, four items were reported less than 50% of the scoping reviews, including description of the rationale for the review (44.6%), accessibility of a protocol and registration information (8.2%), critical appraisal of individual sources of evidence (13.8%), and critical appraisal within sources of evidence (12.5%) (Fig.  4 ).

figure 4

Reporting quality of including reviews

The median score on the PRISMA-ScR checklist was 15 (i.q.r 13–17) and the reviews published in English had significantly higher score than those in Chinese (16 [ 14 , 15 , 16 , 17 , 18 ] vs. 14 [ 13 , 14 , 15 , 16 ]; P  < 0.001). Scoping reviews published in English were more likely than those in Chinese to report the title (95.8% versus 89.0%; P  = 0.009), protocol and registration (13.4% versus 0.0%; P  < 0.001), critical appraisal of individual sources of evidence (17.2% versus 8.4%; P  = 0.014), synthesis of results (79.4% versus 35.1%; P  < 0.001), selection of sources of evidence (90.3% versus 72.7%; P  < 0.001), critical appraisal within sources of evidence (16.4% versus 6.5%; P  = 0.004) and limitations (81.1% versus 32.5%; P  < 0.001). On the other hand, Chinese reviews preferred to report structured summaries (94.2% versus 68.9%; P  < 0.001), eligibility criteria (97.4% versus 86.1%; P  < 0.001), data charting process (75.3% versus 62.6%; P  = 0.009), data items (81.8% versus 71.0%; P  = 0.015) compared to English reviews (Fig.  4 ).

We conducted a comprehensive scoping review of 392 Chinese-authored scoping reviews over the last decade. Our findings revealed a significant increase in the number of scoping reviews in China, particularly since 2020, with 207 scoping reviews published in the past year alone, accounting for approximately half of the total number published in the previous decade. The increase in the number of scoping review publications after 2020 could be attributed to several possible reasons. First, the COVID-19 pandemic in 2020 may have led to a broader literature review in the fields of health sciences and medicine to better understand and respond to the crisis. Second, scoping reviews have gained recognition within the academic and research community, leading to an increasing number of studies adopting scoping reviews for literature synthesis. Third, as research fields continue to evolve and expand, study topics become increasingly complex. Scoping reviews offer a flexible approach covering a wide range of literature, helping researchers to gain a more comprehensive understanding of the current state of research. Finally, some academic institutions and publishers may actively promote the use of scoping review methodology.

The majority of scoping reviews in our studies were conducted in hospitals and universities, with nursing and health being the predominant topics. This trend can be attributed to the education and dissemination efforts of organizations such as Joanna Briggs Institute (JBI) and its collaboration. Additionally, most studies provided a clear definition of scoping reviews and followed the framework by Arksey and O’Malley’ framework [ 14 ]. However, few studies mentioned the dissemination of findings [ 22 ], which may be due to the optional nature in the Asksey and O’Malley framework.

In general, scoping reviews published in English demonstrated higher methodological and reporting quality compared to those published in Chinese. However, some key items recommended by JBI guidance were also poorly reported, including protocol registration, search strategy and data presentation. The significance of the protocol has been emphasized in both the JBI guidance and the PRISMA-ScR checklist (Peters et al., 2022). Surprisingly, less than 10% of the included scoping reviews in our study provided information about the protocol. It is worth noting that various platforms such as Figshare, Open Science Framework, ResearchGate, and Research Square allow protocol registration, and it is encouraged to include full protocols with available information for preregistration purposes.

Unsimilar to systematic review [ 23 ] and other evidence synthesis approaches, scoping reviews have the flexibility to include various types of literature including grey literature, newspapers, websites and social media, to address the question of “what has been done before” [ 24 ]. However, we found a limited number of included scoping reviews that conducted the additional search or consulted with information specialists. Furthermore, among those reviews performed database searches, only half of them provided a comprehensive search strategy for at least one database. It is important to consider conducting a more comprehensive search during the planning phase of a scoping review.

The scoping reviews included in our study showed a preference for presenting results in tables rather than images, resulting in a lack of diversity in data presentation. To enhance the interpretability of scoping review findings, various engaging methods, such as bubble charts, infographics, and Wordless, are available. We recommend that researchers, journal editors, and peer reviewers undergo additional training courses or access online resources (Stern et al., 2018) to improve the methodological quality of scoping reviews conducted by Chinese academic institutions. Furthermore, journal editors should require authors submitting scoping reviews to adhere to the PRISMA-ScR checklist before final submission. Additionally, we encourage research management agencies to promote the practice of conducting a scoping review in their respective fields prior to the initiation of research projects, helping researchers gain a better understanding of their research background and reduce research waste (Khalil et al., 2022).

Our findings are broadly consistent with the two previous scoping reviews of scoping reviews [ 7 , 25 ], demonstrating considerable variability in the purpose, topics, and methodological aspects. However, our study finds certain improvements in specific areas, such as a better understanding of the distinctions between scoping review and systematic review [ 26 , 27 , 28 ], as well as the adoption of descriptive conclusions instead of definitive conclusions for practice.

To identify all relevant scoping reviews conducted by Chinese academic institutions, we conducted a comprehensive search that encompassed grey literature and web search engines. Based on our findings, we provide suggestions to researchers, journal editors, and administrators. However, our study also has some limitations. We may have partially missed some reviews conducted by Chinese academic institutions if the author’s name was not a traditional Chinese form. Additionally, we only included studies in Chinese and English, which may have excluded reviews published in other languages by authors from Chinese academic institutions.

The annual number of scoping reviews conducted by Chinese academic institutions has shown a significant upward trend since 2020, encompassing a wide range of research topics, particularly within the realms of nursing and health. Scoping reviews are increasingly employed in practical applications, such as research preparation and identification of research questions. However, there remains a notable deficiency in the methodological rigor and reporting quality of scoping reviews conducted by Chinese academic institutions. Future research should prioritize enhancing the transparency of search and screening processes, diversifying data presentation techniques, and promoting standardization in reporting practices.

Data availability

The data used to support the findings of this study are included within the article and the supplementary information file.

Moher D, Stewart L, Shekelle P. All in the family: systematic reviews, rapid reviews, scoping reviews, realist reviews, and more. Syst Rev Dec. 2015;22:4:183. https://doi.org/10.1186/s13643-015-0163-7 .

Article   Google Scholar  

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J Jun. 2009;26(2):91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x .

Sutton A, Clowes M, Preston L, Booth A. Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J Sep. 2019;36(3):202–22. https://doi.org/10.1111/hir.12276 .

Colquhoun HL, Levac D, O’Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol Dec. 2014;67(12):1291–4. https://doi.org/10.1016/j.jclinepi.2014.03.013 .

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci Sep. 2010;20:5:69. https://doi.org/10.1186/1748-5908-5-69 .

Davis K, Drey N, Gould D. What are scoping studies? A review of the nursing literature. Int J Nurs Stud Oct. 2009;46(10):1386–400. https://doi.org/10.1016/j.ijnurstu.2009.02.010 .

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol Feb. 2016;9:16:15. https://doi.org/10.1186/s12874-016-0116-4 .

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol Nov. 2018;19(1):143. https://doi.org/10.1186/s12874-018-0611-x .

G MDJPC, BS PMC, K H. D. P. Chapter 11: scoping reviews. The Joanna Briggs Institute. Accessed April 1, 2020. https://reviewersmanual.joannabriggs.org .

Daudt HM, van Mossel C, Scott SJ. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. BMC Med Res Methodol. Mar 2013;23:13:48. https://doi.org/10.1186/1471-2288-13-48 .

Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Syst Rev Jun. 2012;9:1:28. https://doi.org/10.1186/2046-4053-1-28 .

Harms MC, Goodwin VA. Scoping reviews. Physiotherapy Dec. 2019;105(4):397–8. https://doi.org/10.1016/j.physio.2019.10.005 .

Article   CAS   Google Scholar  

Mays N, Popay RE. J. Studying the organization and delivery of health services: research methods. In: Fulop. N, Allen. P, Clarke. A, Black. N, eds. Synthesising research evidence . Routledge; 2001:194.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32. https://doi.org/10.1080/1364557032000119616 .

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc Sep. 2015;13(3):141–6. https://doi.org/10.1097/xeb.0000000000000050 .

Peters MDJ, Marnie C, Tricco AC, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth Oct. 2020;18(10):2119–26. https://doi.org/10.11124/jbies-20-00167 .

Peters MDJ, Marnie C, Tricco AC, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Implement Mar. 2021;19(1):3–10. https://doi.org/10.1097/xeb.0000000000000277 .

Tricco AC, Lillie E, Zarin W, et al. PRISMA Extension for scoping reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med Oct. 2018;2(7):467–73. https://doi.org/10.7326/m18-0850 .

Xiyi W, Zhihong Y, Leiwen T. An integrative review of scoping review applied in nursing iterature. Chin J Nurs. 2019;54(08):1259–63.

Google Scholar  

Mu F, Tang M, Guan Y, et al. Knowledge mapping of the Links between the gut microbiota and heart failure: a Scientometric Investigation (2006–2021). Front Cardiovasc Med. 2022;9:882660. https://doi.org/10.3389/fcvm.2022.882660 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Lin Z, Ji X, Tian N, Gan Y, Ke L. Mapping intellectual structure for the long non-coding RNA in Hepatocellular Carcinoma Development Research. Front Genet. 2021;12:771810. https://doi.org/10.3389/fgene.2021.771810 .

Article   CAS   PubMed   Google Scholar  

Straus SE, Tetroe JM, Graham ID. Knowledge translation is the use of knowledge in health care decision making. J Clin Epidemiol Jan. 2011;64(1):6–10. https://doi.org/10.1016/j.jclinepi.2009.08.016 .

Higgins J, Thomas J, Chandler J et al. Cochrane Handbook for Systematic Reviews of Interventions. http://handbook.cochrane.org/ .

Khalil H, Peters MD, Tricco AC, et al. Conducting high quality scoping reviews-challenges and solutions. J Clin Epidemiol Feb. 2021;130:156–60. https://doi.org/10.1016/j.jclinepi.2020.10.009 .

Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods Dec. 2014;5(4):371–85. https://doi.org/10.1002/jrsm.1123 .

Qian J, Sun S, Wang M, Yu X. Nonpharmacological pain management interventions in medical and surgical abortion: a scoping review. Int J Nurs Pract Apr. 2022;3:e13056. https://doi.org/10.1111/ijn.13056 .

Wang Y, Wang Z, Liu G, et al. Application of Serious games in Health Care: scoping review and bibliometric analysis. Front Public Health. 2022;10:896974. https://doi.org/10.3389/fpubh.2022.896974 .

Article   PubMed   PubMed Central   Google Scholar  

Huang L, Chang H, Peng X, Zhang F, Mo B, Liu Y. Formally reporting incidents of workplace violence among nurses: a scoping review. J Nurs Manag Sep. 2022;30(6):1677–87. https://doi.org/10.1111/jonm.13567 .

Download references

This research was funded by the Sichuan Provincial Department of Science and Technology Project, grant number 2023NSFSC0520. The funder had no role in the design of the study; in the collection, analysis, or interpretation of the data; in writing or approving the manuscript; or in the decision to submit the manuscript for publication.

Author information

Authors and affiliations.

Department of Clinical Nutrition, West China Hospital, Sichuan University, Chengdu, 610041, China

Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, Chengdu, 610041, China

Xinyu Xue, Xintong Tang, Shanshan Liu, Ningsu Chen & Jiajie Yu

Evidence-based Nursing Center, West China School of Nursing, West China Hospital, Sichuan University, Chengdu, 610041, China

Ting Yu & Zhonglan Chen

You can also search for this author in PubMed   Google Scholar

Contributions

Xue Xinyu: searched the literature, extracted data, synthesized data, visualization and drafted the manuscript. Tang Xintong: searched the literature and extracted data. Liu Shanshan: searched the literature and extracted data. Yu Ting: searched the literature and extracted data. Chen Zhonglan: searched the literature and extracted data. Chen Ningsu: searched the literature and extracted data. Yu Jiajie: designed the systematic review and revised the manuscript.

Corresponding author

Correspondence to Jiajie Yu .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

All authors reported no potential conflicts of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Xue, X., Tang, X., Liu, S. et al. A scoping review on the methodological and reporting quality of scoping reviews in China. BMC Med Res Methodol 24 , 45 (2024). https://doi.org/10.1186/s12874-024-02172-y

Download citation

Received : 15 August 2023

Accepted : 05 February 2024

Published : 22 February 2024

DOI : https://doi.org/10.1186/s12874-024-02172-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Scoping review
  • Literature review
  • Methodology

BMC Medical Research Methodology

ISSN: 1471-2288

an updated review of the literature

Breakfast consumption and adiposity among children and adolescents: an updated review of the literature

Affiliations.

  • 1 Friedman School of Nutrition Science and Policy, ChildObesity180, Tufts University, Boston, USA. [email protected].
  • 2 Friedman School of Nutrition Science and Policy, ChildObesity180, Tufts University, Boston, USA.
  • PMID: 26842913
  • DOI: 10.1111/ijpo.12082

Background: Breakfast consumption has been associated with reduced risk of overweight and obesity among children, but previous evidence reviews fail to confirm a causal relationship.

Objectives: To review recent literature on breakfast consumption and adiposity among children and discuss potential underlying mechanisms.

Methods: A comprehensive literature search of studies published since the 2010 US National Evidence Library review (January 2010-January 2015) was conducted.

Results: Twelve studies met inclusion criteria. All were conducted in industrialized countries: six in Europe, four in the USA, one in China and one in Australia. Ten of the studies used observational longitudinal designs, with follow-up periods ranging from 1 to 27 years (median: 3, mean: 7.4); of these, eight reported inverse associations between breakfast consumption and excess adiposity, while two found no association. The other studies (1 case-control, 1 experimental) each reported a protective effect of breakfast consumption on overweight and obesity among children.

Conclusions: Findings corroborate results from previous reviews, adding support for a possible, protective role for breakfast consumption in preventing excess adiposity during childhood and adolescence. However, drawing a causal conclusion from the collective evidence is curtailed by methodological limitations and inconsistencies, including study design, follow-up duration and frequency, exposure and outcome assessment, as well as limited consideration of confounding, mediating and effect-modifying variables. More rigorous study designs employing valid and standardized measurement of relevant variables are needed.

Keywords: Breakfast; children; obesity; overweight.

© 2016 World Obesity.

Publication types

  • Adiposity / physiology*
  • Body Mass Index
  • Breakfast / physiology*
  • Outcome Assessment, Health Care
  • Overweight / prevention & control*
  • Pediatric Obesity / prevention & control*
  • United States

IMAGES

  1. How To Write A Literature Review

    an updated review of the literature

  2. 50 Smart Literature Review Templates (APA) ᐅ TemplateLab

    an updated review of the literature

  3. Good literature review sample. Bad, Better, Best Examples of Literature

    an updated review of the literature

  4. Steps of conducting a Literature Review

    an updated review of the literature

  5. How to Write a Stellar Literature Review

    an updated review of the literature

  6. literature review article examples Sample of research literature review

    an updated review of the literature

VIDEO

  1. Review of literature

  2. Literature Review

  3. Part 2 Writing the Review of Literature

  4. Writing a Literature Review

  5. Chapter 5 Review of Literature PART 02

  6. Effective Review of Literature

COMMENTS

  1. Early Childhood Educators' Well-Being: An Updated Review of the Literature

    Attention is given to ways recent studies address concerns raised in a review of earlier literature (Hall-Kenyon et al. in Early Child Educ J 42(3):153-162, 2014, doi: 10.1007/s10643-013-0595-4 ), and what implications recent studies have for future research efforts concerned with educators' well-being.

  2. Genetics of adult attachment: An updated review of the literature

    A narrative review is conducted analyzing the existing literature regarding the implication of candidate genes related to oxytocin, dopaminergic pathways, serotonergic pathways and brain-derived neurotrophic factor in adult attachment, with both vulnerability and differential susceptibility approaches, yielding mixed results.

  3. Condylar hyperplasia: An updated review of the literature

    Condylar hyperplasia (CH) is a rare disorder characterized by excessive bone growth that usually presents unilaterally, resulting in facial asymmetry. 1 Its etiology and pathogenesis are poorly understood. Facial asymmetry is often the reason that patients present for treatment of the disorder. In many cases, occlusal discrepancies and ...

  4. An Updated Review of the Literature on LGBTQ+ Intimate ...

    An Updated Review of the Literature on LGBTQ+ Intimate Partner Violence Sexual Orientation and Identity (E Coleman and J Vencill, Section Editors) Published: 17 September 2018 Volume 10 , pages 265-272, ( 2018 ) Cite this article Download PDF Current Sexual Health Reports Aims and scope Submit manuscript Melissa Decker, Heather L. Littleton &

  5. The mass customization decade: An updated review of the literature

    The result is a concise compendium of the relevant literature produced on the topic in the past decade. The study reviews the literature on mass customization over the last decade. This is an update of the classical study by Da Silveira et al. (2001). The study provides a conceptual framework to support future research. We review the concept ...

  6. Compassion in healthcare: an updated scoping review of the literature

    Given the large influx and the disparate nature of the topic within the healthcare literature over the past 5 years, the objective of this study was to provide an update to our original scoping review to provide a current and comprehensive map of the literature to guide future research and to identify gaps and limitations that remain unaddressed.

  7. PDF Early Childhood Educators' Well-Being: An Updated Review of the Literature

    Introduction. Researchers across many national contexts are increasingly recognising the connections between early childhood edu-cators' well-being and their capacity for providing high quality education and care. Hall-Kenyon et al.'s (2014) review of literature spanning more than 30 years, identified. Tamara Cumming [email protected].

  8. When and how to update systematic reviews: consensus and checklist

    Updating of systematic reviews is generally more efficient than starting all over again when new evidence emerges, but to date there has been no clear guidance on how to do this. This guidance helps authors of systematic reviews, commissioners, and editors decide when to update a systematic review, and then how to go about updating the review. Systematic reviews synthesise relevant research ...

  9. Chapter IV: Updating a review

    Garner and colleagues define an update as "a new edition of a published systematic review with changes that can include new data, new methods, or new analyses to the previous edition" (Garner et al 2016). Adding new studies and new data can substantively change the findings of the review.

  10. The nature of labour pain: An updated review of the literature

    Lowe's review summarised the literature regarding labour pain up until the year 2000, and interpreted this literature based on pain theories at the time. Over the past two decades shifts have occurred in both the characteristics of labouring women, the labour environment, and pain science. This present review aims to provide an update of our ...

  11. Self-report measures of boredom: An updated review of the literature

    Self-report measures of boredom: An updated review of the literature. The Journal of Psychology: Interdisciplinary and Applied, 150 (2), 196-228. https:// https://doi.org/10.1080/00223980.2015.1074531 Abstract A detailed review of the psychometric measures of boredom was published approximately 12 years ago (Vodanovich, 2003).

  12. When and how to update systematic reviews

    The updating strategies were: a) steps in maintaining an updated review (Chalmers 1993; Higgins 2005), b) assessment of the need to update ... 3. "literature review"/ 4. (systematic review$ or cumulative meta‐analys$ or hta or ((clinical or prevent$) adj2 guideline$)).mp.

  13. Homophobia: an updated review of the literature: Sexual and

    This paper updates the literature review on homophobia revisiting society's assumptions about homosexuals. Changes are discussed in the context of definitions of homophobia, socio-cultural and politico-legal perspectives and homophobia and medicine. New current assumptions are drawn up and offered as a way of understanding both development and ...

  14. What is a literature review?

    A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations.

  15. The mass customization decade: An updated review of the literature

    The study reviews the literature on mass customization over the last decade. This is an update of the classical study by Da Silveira et al. (2001). The study provides a conceptual framework to support future research. We review the concept, economics, success factors, and enablers of mass customization.

  16. Homophobia: An updated review of the literature.

    Homophobia: An updated review of the literature. Citation Ahmad, S., & Bhugra, D. (2010). Homophobia: An updated review of the literature. Sexual and Relationship Therapy, 25 (4), 447-455. https:// https://doi.org/10.1080/14681994.2010.515206 Abstract

  17. How to Write a Literature Review

    What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic.

  18. Self-Report Measures of Boredom: An Updated Review of the Literature

    Given these assessment advancements, an updated review of self-report boredom scales is warranted. The primary focus of the current review is research published since 2003, and it includes a total ...

  19. 'It depends': what 86 systematic reviews tell us about what strategies

    The objective of this study was to update and extend two previous reviews of systematic reviews of strategies designed to implement research evidence into clinical practice. We developed a comprehensive systematic literature search strategy based on the terms used in the previous reviews to identify studies that looked explicitly at ...

  20. Condylar hyperplasia: An updated review of the literature

    Condylar hyperplasia: An updated review of the literature Korean J Orthod. 2015 Nov;45 (6):333-40. doi: 10.4041/kjod.2015.45.6.333. Epub 2015 Nov 20. Authors Luis Eduardo Almeida 1 , Joseph Zacharias 1 , Sean Pierce 1 Affiliation

  21. Self-Report Measures of Boredom: An Updated Review of the Literature

    Publication types Review MeSH terms Attention Boredom* Humans Psychometrics / methods* Self Report* A detailed review of the psychometric measures of boredom was published approximately 12 years ago (Vodanovich, 2003). Since that time, numerous studies have been conducted on existing scales, and new measures of boredom have been developed.

  22. Suicide in rural areas: An updated review of the literature.

    Suicide in rural areas: An updated review of the literature. Journal of Rural Mental Health, 38 (2), 65-78. https:// https://doi.org/10.1037/rmh0000018 Abstract Suicide is a significant public health concern at a global level and occurs at a greater rate in rural compared with urban areas.

  23. Whole-Body Cryotherapy in Athletes: From Therapy to Stimulation. An

    This review article collects the most recent literature (since 2010, Banfi et al., 2010b) on whole-body cryotherapy with the purpose of delivering a complete and updated overview of the newest findings and the directions taken in research in this field. In particular, given the high number of new scientific findings mostly associated with great ...

  24. Types of reviews

    Main purpose is to map out and categorize existing literature, identify gaps in literature; Search comprehensiveness determined by time/scope constraints, could take longer than a systematic review; No formal quality assessment or critical appraisal; Learn more about the methods:

  25. A scoping review on the methodological and reporting quality of scoping

    Reviews published in English were more likely to search the grey literature (P = 0.005), consult information specialists (P < 0.001) and conduct an updated search (P = 0.012) than those in Chinese. Reviews published in English had a significantly high score compared to those published in Chinese (16 vs. 14; P < 0.001).

  26. Diagnostics

    Digitalizing all aspects of dental care is a contemporary approach to ensuring the best possible clinical outcomes. Ongoing advancements in 3D face acquisition have been driven by continuous research on craniofacial structures and treatment effects. An array of 3D surface-imaging systems are currently available for generating photorealistic 3D facial images. However, choosing a purpose ...

  27. Acute hepatitis of unknown origin in children: Should we move the

    Dear Editor. We read with interest the article by Dr. Grand which appeared in a recent issue of the Journal, where he reviews in great detail the updated literature on a) the general features of human adenoviruses (HAdVs) and adeno-associated viruses type 2 (AAV2), and b) these two viruses' possible association with the last year paediatric outbreak of severe acute hepatitis of unknown ...

  28. Breakfast consumption and adiposity among children and ...

    Breakfast consumption and adiposity among children and adolescents: an updated review of the literature Findings corroborate results from previous reviews, adding support for a possible, protective role for breakfast consumption in preventing excess adiposity during childhood and adolescence.

  29. IFAC and AICPA & CIMA study observes more detail in sustainability

    The study, which is an annual benchmark that has now been updated for 2022 data, also shows that the companies included in the study are obtaining a greater scope of assurance on their sustainability disclosures. ... IFAC releases literature review to support sustainability reporting. 10 Jan 2024. IFAC study on sustainability disclosure beyond ...