literature review of secondary research

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Teesside University Student & Library Services
• Learning Hub Group

Research Methods

Secondary research.

• Primary Research

What is Secondary Research?

Advantages and disadvantages of secondary research, secondary research in literature reviews, secondary research - going beyond literature reviews, main stages of secondary research, useful resources, using material on this page.

• Quantitative Research This link opens in a new window
• Qualitative Research This link opens in a new window
• Being Critical This link opens in a new window
• Subject LibGuides This link opens in a new window

Secondary research

Secondary research uses research and data that has already been carried out. It is sometimes referred to as desk research. It is a good starting point for any type of research as it enables you to analyse what research has already been undertaken and identify any gaps. 

You may only need to carry out secondary research for your assessment or you may need to use secondary research as a starting point, before undertaking your own primary research .

Searching for both primary and secondary sources can help to ensure that you are up to date with what research has already been carried out in your area of interest and to identify the key researchers in the field.

"Secondary sources are the books, articles, papers and similar materials written or produced by others that help you to form your background understanding of the subject. You would use these to find out about experts’ findings, analyses or perspectives on the issue and decide whether to draw upon these explicitly in your research." (Cottrell, 2014, p. 123).

Examples of secondary research sources include:.

• journal articles
• official statistics, such as government reports or organisations which have collected and published data

Primary research  involves gathering data which has not been collected before. Methods to collect it can include interviews, focus groups, controlled trials and case studies. Secondary research often comments on and analyses this primary research.

Gopalakrishnan and Ganeshkumar (2013, p. 10) explain the difference between primary and secondary research:

"Primary research is collecting data directly from patients or population, while secondary research is the analysis of data already collected through primary research. A review is an article that summarizes a number of primary studies and may draw conclusions on the topic of interest which can be traditional (unsystematic) or systematic".

Secondary Data

As secondary data has already been collected by someone else for their research purposes, it may not cover all of the areas of interest for your research topic. This research will need to be analysed alongside other research sources and data in the same subject area in order to confirm, dispute or discuss the findings in a wider context.

"Secondary source data, as the name infers, provides second-hand information. The data come ‘pre-packaged’, their form and content reflecting the fact that they have been produced by someone other than the researcher and will not have been produced specifically for the purpose of the research project. The data, none the less, will have some relevance for the research in terms of the information they contain, and the task for the researcher is to extract that information and re-use it in the context of his/her own research project." (Denscombe, 2021, p. 268)

In the video below Dr. Benedict Wheeler (Senior Research Fellow at the European Center for Environment and Human Health at the University of Exeter Medical School) discusses secondary data analysis. Secondary data was used for his research on how the environment affects health and well-being and utilising this secondary data gave access to a larger data set.

As with all research, an important part of the process is to critically evaluate any sources you use. There are tools to help with this in the  Being Critical  section of the guide.

Louise Corti, from the UK Data Archive, discusses using secondary data  in the video below. T he importance of evaluating secondary research is discussed - this is to ensure the data is appropriate for your research and to investigate how the data was collected.

There are advantages and disadvantages to secondary research:

Advantages:

• Usually low cost
• Easily accessible
• Provides background information to clarify / refine research areas
• Increases breadth of knowledge
• Shows different examples of research methods
• Can highlight gaps in the research and potentially outline areas of difficulty
• Can incorporate a wide range of data
• Allows you to identify opposing views and supporting arguments for your research topic
• Highlights the key researchers and work which is being undertaken within the subject area
• Helps to put your research topic into perspective

Disadvantages

• Can be out of date
• Might be unreliable if it is not clear where or how the research has been collected - remember to think critically
• May not be applicable to your specific research question as the aims will have had a different focus

Literature reviews 

Secondary research for your major project may take the form of a literature review . this is where you will outline the main research which has already been written on your topic. this might include theories and concepts connected with your topic and it should also look to see if there are any gaps in the research., as the criteria and guidance will differ for each school, it is important that you check the guidance which you have been given for your assessment. this may be in blackboard and you can also check with your supervisor..

The videos below include some insights from academics regarding the importance of literature reviews.

Secondary research which goes beyond literature reviews

For some dissertations/major projects there might only be a literature review (discussed above ). For others there could be a literature review followed by primary research and for others the literature review might be followed by further secondary research. 

You may be asked to write a literature review which will form a background chapter to give context to your project and provide the necessary history for the research topic. However, you may then also be expected to produce the rest of your project using additional secondary research methods, which will need to produce results and findings which are distinct from the background chapter t o avoid repetition .

Remember, as the criteria and guidance will differ for each School, it is important that you check the guidance which you have been given for your assessment. This may be in Blackboard and you can also check with your supervisor.

Although this type of secondary research will go beyond a literature review, it will still rely on research which has already been undertaken. And,  "just as in primary research, secondary research designs can be either quantitative, qualitative, or a mixture of both strategies of inquiry" (Manu and Akotia, 2021, p. 4) .

Your secondary research may use the literature review to focus on a specific theme, which is then discussed further in the main project. Or it may use an alternative approach. Some examples are included below.  Remember to speak with your supervisor if you are struggling to define these areas.

Some approaches of how to conduct secondary research include:

• A systematic review is a structured literature review that involves identifying all of the relevant primary research using a rigorous search strategy to answer a focused research question.
• This involves comprehensive searching which is used to identify themes or concepts across a number of relevant studies. 
• The review will assess the q uality of the research and provide a summary and synthesis of all relevant available research on the topic.
• The systematic review  LibGuide goes into more detail about this process (The guide is aimed a PhD/Researcher students. However, students on other levels of study may find parts of the guide helpful too).
• Scoping reviews aim to identify and assess available research on a specific topic (which can include ongoing research). 
• They are "particularly useful when a body of literature has not yet been comprehensively reviewed, or exhibits a complex or heterogeneous nature not amenable to a more precise systematic review of the evidence. While scoping reviews may be conducted to determine the value and probable scope of a full systematic review, they may also be undertaken as exercises in and of themselves to summarize and disseminate research findings, to identify research gaps, and to make recommendations for the future research."  (Peters et al., 2015) .
• This is designed to  summarise the current knowledge and provide priorities for future research.
• "A state-of-the-art review will often highlight new ideas or gaps in research with no official quality assessment." (Baguss, 2020) .
• "Bibliometric analysis is a popular and rigorous method for exploring and analyzing large volumes of scientific data." (Donthu et al., 2021)
• Quantitative methods and statistics are used to analyse the bibliographic data of published literature. This can be used to measure the impact of authors, publications, or topics within a subject area.

The bibliometric analysis often uses the data from a citation source such as Scopus or Web of Science .

• This is a technique used to combine the statistic results of prior quantitative studies in order to increase precision and validity.
• "It goes beyond the parameters of a literature review, which assesses existing literature, to actually perform calculations based on the results collated, thereby coming up with new results" (Curtis and Curtis, 2011, p. 220)

(Adapted from: Grant and Booth, 2009, cited in Sarhan and Manu, 2021, p. 72 )

• Grounded Theory is used to create explanatory theory from data which has been collected.
• "Grounded theory data analysis strategies can be used with different types of data, including secondary data." ( Whiteside, Mills and McCalman, 2012 )
• This allows you to use a specific theory or theories which can then be applied to your chosen topic/research area.
• You could focus on one case study which is analysed in depth, or you could examine more than one in order to compare and contrast the important aspects of your research question.
• "Good case studies often begin with a predicament that is poorly comprehended and is inadequately explained or traditionally rationalised by numerous conflicting accounts. Therefore, the aim is to comprehend an existent problem and to use the acquired understandings to develop new theoretical outlooks or explanations."  ( Papachroni and Lochrie, 2015, p. 81 )

Main stages of secondary research for a dissertation/major project

In general, the main stages for conducting secondary research for your dissertation or major project will include:

Click on the image below to access the reading list which includes resources used in this guide as well as some additional useful resources.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License .

• << Previous: Primary Research
• Next: Quantitative Research >>
• Last Updated: Aug 11, 2022 3:41 PM
• URL: https://libguides.tees.ac.uk/researchmethods

Understanding research and critical appraisal

• Introduction

What is secondary research?

Secondary research study designs.

• Primary research
• Critical appraisal of research papers
• Useful terminology
• Further reading and helpful resources

The aim of secondary research is to produce a more or less systematic appraisal and/or synthesis of the existing primary research on a topic. There are numerous types of reviews which aim to summarise or synthesise the evidence on a topic, but here we will focus on two: meta-analyses and systematic reviews.

For a fuller discussion of the range of review types, their features and uses, see: Sutton, A. et al . (2019) 'Meeting the review family: exploring review types and associated information retrieval requirements', Health Information and Libraries Journal , 36 (3), pp. 202-222.  doi:10.1111/hir.12276

Meta-analysis

A meta-analysis is a statistical synthesis of the results from multiple individual studies, usually randomised controlled trials (RCTs),

Carrying out a meta-analysis of studies allows results from multiple studies looking at the effect of an intervention to be combined, allowing for greater precision in the estimation of effects, and clarity over the direction and size of an effect. A meta-analysis can provide more conclusive evidence for or against the effectiveness of an intervention than individual studies alone.

A good meta-analysis should always be based on a systematic review of studies, and requires some homogeneity of participants, settings, interventions and outcome measures in the studies included.

Systematic review

A systematic review is not simply a literature review. A systematic review is a study which aims to synthesise all of the available primary research on a specific topic. The first step in a systematic review is a thorough search of all appropriate sources, including subject related databases, clinical trial registers and grey literature, in order to identify all of the relevant evidence. These searches should ideally be carried out by a librarian or information specialist in the field, or by others with a similar level of expertise. The systematic review itself should be carried out by two or more researchers, as a means of reducing possible bias.

All identified studies are screened for inclusion or exclusion according to strict criteria set out at the start of the study, and the data from those studies selected for inclusion is analysed and synthesised. Part of this process is an attempt to identify any potential source of bias in existing findings. A systematic review will offer a summary of the available research findings, and offer conclusions on the basis of these, taking into account any flaws or limitations in the original studies.

A systematic review can offer more generalisability and consistency of research findings than the individual studies on which it is based.

Systematic reviews may employ quantitative, qualitative (experiential), or mixed-methods approaches.

• << Previous: Introduction
• Next: Primary research >>
• Last Updated: Mar 26, 2024 4:38 PM
• URL: https://libguides.sgul.ac.uk/researchdesign

Secondary Research Guide: Definition, Methods, Examples

Apr 3, 2024

8 min. read

The internet has vastly expanded our access to information, allowing us to learn almost anything about everything. But not all market research is created equal , and this secondary research guide explains why.

There are two key ways to do research. One is to test your own ideas, make your own observations, and collect your own data to derive conclusions. The other is to use secondary research — where someone else has done most of the heavy lifting for you. 

Here’s an overview of secondary research and the value it brings to data-driven businesses.

Secondary Research Definition: What Is Secondary Research?

Primary vs Secondary Market Research

What Are Secondary Research Methods?

Advantages of secondary research, disadvantages of secondary research, best practices for secondary research, how to conduct secondary research with meltwater.

Secondary research definition: The process of collecting information from existing sources and data that have already been analyzed by others.

Secondary research (aka desk research ) provides a foundation to help you understand a topic, with the goal of building on existing knowledge. They often cover the same information as primary sources, but they add a layer of analysis and explanation to them.

Users can choose from several secondary research types and sources, including:

• Journal articles
• Research papers

With secondary sources, users can draw insights, detect trends , and validate findings to jumpstart their research efforts.

Primary vs. Secondary Market Research

We’ve touched a little on primary research , but it’s essential to understand exactly how primary and secondary research are unique.

Think of primary research as the “thing” itself, and secondary research as the analysis of the “thing,” like these primary and secondary research examples:

• An expert gives an interview (primary research) and a marketer uses that interview to write an article (secondary research).
• A company conducts a consumer satisfaction survey (primary research) and a business analyst uses the survey data to write a market trend report (secondary research).
• A marketing team launches a new advertising campaign across various platforms (primary research) and a marketing research firm, like Meltwater for market research , compiles the campaign performance data to benchmark against industry standards (secondary research).

In other words, primary sources make original contributions to a topic or issue, while secondary sources analyze, synthesize, or interpret primary sources.

Both are necessary when optimizing a business, gaining a competitive edge , improving marketing, or understanding consumer trends that may impact your business.

Secondary research methods focus on analyzing existing data rather than collecting primary data . Common examples of secondary research methods include:

• Literature review . Researchers analyze and synthesize existing literature (e.g., white papers, research papers, articles) to find knowledge gaps and build on current findings.
• Content analysis . Researchers review media sources and published content to find meaningful patterns and trends.
• AI-powered secondary research . Platforms like Meltwater for market research analyze vast amounts of complex data and use AI technologies like natural language processing and machine learning to turn data into contextual insights.

Researchers today have access to more market research tools and technology than ever before, allowing them to streamline their efforts and improve their findings.

Want to see how Meltwater can complement your secondary market research efforts? Simply fill out the form at the bottom of this post, and we'll be in touch.

Conducting secondary research offers benefits in every job function and use case, from marketing to the C-suite. Here are a few advantages you can expect.

Cost and time efficiency

Using existing research saves you time and money compared to conducting primary research. Secondary data is readily available and easily accessible via libraries, free publications, or the Internet. This is particularly advantageous when you face time constraints or when a project requires a large amount of data and research.

Access to large datasets

Secondary data gives you access to larger data sets and sample sizes compared to what primary methods may produce. Larger sample sizes can improve the statistical power of the study and add more credibility to your findings.

Ability to analyze trends and patterns

Using larger sample sizes, researchers have more opportunities to find and analyze trends and patterns. The more data that supports a trend or pattern, the more trustworthy the trend becomes and the more useful for making decisions. 

Historical context

Using a combination of older and recent data allows researchers to gain historical context about patterns and trends. Learning what’s happened before can help decision-makers gain a better current understanding and improve how they approach a problem or project.

Basis for further research

Ideally, you’ll use secondary research to further other efforts . Secondary sources help to identify knowledge gaps, highlight areas for improvement, or conduct deeper investigations.

Tip: Learn how to use Meltwater as a research tool and how Meltwater uses AI.

Secondary research comes with a few drawbacks, though these aren’t necessarily deal breakers when deciding to use secondary sources.

Reliability concerns

Researchers don’t always know where the data comes from or how it’s collected, which can lead to reliability concerns. They don’t control the initial process, nor do they always know the original purpose for collecting the data, both of which can lead to skewed results.

Potential bias

The original data collectors may have a specific agenda when doing their primary research, which may lead to biased findings. Evaluating the credibility and integrity of secondary data sources can prove difficult.

Outdated information

Secondary sources may contain outdated information, especially when dealing with rapidly evolving trends or fields. Using outdated information can lead to inaccurate conclusions and widen knowledge gaps.

Limitations in customization

Relying on secondary data means being at the mercy of what’s already published. It doesn’t consider your specific use cases, which limits you as to how you can customize and use the data.

A lack of relevance

Secondary research rarely holds all the answers you need, at least from a single source. You typically need multiple secondary sources to piece together a narrative, and even then you might not find the specific information you need.

To make secondary market research your new best friend, you’ll need to think critically about its strengths and find ways to overcome its weaknesses. Let’s review some best practices to use secondary research to its fullest potential.

Identify credible sources for secondary research

To overcome the challenges of bias, accuracy, and reliability, choose secondary sources that have a demonstrated history of excellence . For example, an article published in a medical journal naturally has more credibility than a blog post on a little-known website.

Assess credibility based on peer reviews, author expertise, sampling techniques, publication reputation, and data collection methodologies. Cross-reference the data with other sources to gain a general consensus of truth.

The more credibility “factors” a source has, the more confidently you can rely on it. 

Evaluate the quality and relevance of secondary data

You can gauge the quality of the data by asking simple questions:

• How complete is the data? 
• How old is the data? 
• Is this data relevant to my needs?
• Does the data come from a known, trustworthy source?

It’s best to focus on data that aligns with your research objectives. Knowing the questions you want to answer and the outcomes you want to achieve ahead of time helps you focus only on data that offers meaningful insights.

Document your sources 

If you’re sharing secondary data with others, it’s essential to document your sources to gain others’ trust. They don’t have the benefit of being “in the trenches” with you during your research, and sharing your sources can add credibility to your findings and gain instant buy-in.

Secondary market research offers an efficient, cost-effective way to learn more about a topic or trend, providing a comprehensive understanding of the customer journey . Compared to primary research, users can gain broader insights, analyze trends and patterns, and gain a solid foundation for further exploration by using secondary sources.

Meltwater for market research speeds up the time to value in using secondary research with AI-powered insights, enhancing your understanding of the customer journey. Using natural language processing, machine learning, and trusted data science processes, Meltwater helps you find relevant data and automatically surfaces insights to help you understand its significance. Our solution identifies hidden connections between data points you might not know to look for and spells out what the data means, allowing you to make better decisions based on accurate conclusions. Learn more about Meltwater's power as a secondary research solution when you request a demo by filling out the form below:

Continue Reading

How To Do Market Research: Definition, Types, Methods

What Are Consumer Insights? Meaning, Examples, Strategy

Market Intelligence 101: What It Is & How To Use It

The 13 Best Market Research Tools

Consumer Intelligence: Definition & Examples

9 Top Consumer Insights Tools & Companies

What Is Desk Research? Meaning, Methodology, Examples

Research Basics

• What Is Research?
• Types of Research
• Secondary Research | Literature Review
• Developing Your Topic
• Primary vs. Secondary Sources
• Evaluating Sources
• Responsible Conduct of Research
• Additional Help

What is Secondary Research?

Secondary research, also known as a literature review , preliminary research , historical research , background research , desk research , or library research , is research that analyzes or describes prior research. Rather than generating and analyzing new data, secondary research analyzes existing research results to establish the boundaries of knowledge on a topic, to identify trends or new practices, to test mathematical models or train machine learning systems, or to verify facts and figures. Secondary research is also used to justify the need for primary research as well as to justify and support other activities. For example, secondary research may be used to support a proposal to modernize a manufacturing plant, to justify the use of newly a developed treatment for cancer, to strengthen a business proposal, or to validate points made in a speech.

The following guides, published by the library, offer more information on how to do secondary research or a literature review:

• << Previous: Types of Research
• Next: Developing Your Topic >>
• Last Updated: Dec 21, 2023 3:49 PM
• URL: https://guides.library.iit.edu/research_basics

Library Research Skills for Biologists

• Choose your own research adventure
• The cycle of scholarly communication
• Primary literature

Secondary literature

• Tertiary literature
• QUIZ 01: Stages of scholarly communication
• Popular writing
• Popular science writing
• Scholarly writing
• Quiz 02: Activity Break
• Keywords = search terms
• ACTIVITY BREAK - Identify search terms
• Combining concepts
• Phrase searching
• Quiz 03: Boolean operators
• What is Summon?
• Refine your search
• Reference books
• Finding a book from a citation
• Print book location
• Using print books effectively
• What if an article is not available at UBC?
• Why use advanced tools?
• Article indexes
• Subject headings
• Web of Science
• Finding government information using Summon
• Finding government information using Google
• Research Guides
• What is plagiarism?
• Common knowledge
• Paraphrasing vs. summarizing
• What is citing?
• How to cite your sources
• Tools for managing research literature
• Evaluating sources
• Evaluation tools

How can I use this?

Secondary Sources are useful for both gaining a broad knowledge of a topic area (e.g review papers), and for understanding how certain discoveries or projects were received by the scholarly community. They give you opinions, analysis, and a discussion of impact which you can use to place primary sources in context.

What is secondary literature?

When other writers summarize or review the theories and results of original scientific research, their publications are known as Secondary Literature . These publication types serve to synthesize the findings to date and present the ideas to a wider audience. Examples of secondary literature include:

• Review articles
• Edited volumes
• Books summarizing research in an area
• Abstracts and Indexes - Tools used to find articles in scholarly journals

Secondary literature is not as current as the primary literature but it is often more descriptive and is useful for finding introductory material.

Insulin: Discussing the experiment and finding papers

References cited.

[1] Grodsky, G.M. 1970. Insulin and the pancreas. Vitamins & Hormones , 28: 37-101.

• << Previous: Primary literature
• Next: Tertiary literature >>
• Last Updated: Mar 1, 2024 3:23 PM
• URL: https://guides.library.ubc.ca/tutorial-biology

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Secondary Sources
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

In the social sciences, a secondary source is usually a scholar book, journal article, or digital or print document that was created by someone who did not directly experience or participate in the events or conditions under investigation. Secondary sources are not evidence per se, but rather, provide an interpretation, analysis, or commentary derived from the content of primary source materials and/or other secondary sources.

Value of Secondary Sources

To do research, you must cite research. Primary sources do not represent research per se, but only the artifacts from which most research is derived. Therefore, the majority of sources in a literature review are secondary sources that present research findings, analysis, and the evaluation of other researcher's works.

Reviewing secondary source material can be of valu e in improving your overall research paper because secondary sources facilitate the communication of what is known about a topic. This literature also helps you understand the level of uncertainty about what is currently known and what additional information is needed from further research. It is important to note, however, that secondary sources are not the subject of your analysis. Instead, they represent various opinions, interpretations, and arguments about the research problem you are investigating--opinions, interpretations, and arguments with which you may either agree or disagree with as part of your own analysis of the literature.

Examples of secondary sources you could review as part of your overall study include:     * Bibliographies [also considered tertiary]     * Biographical works     * Books, other than fiction and autobiography     * Commentaries, criticisms     * Dictionaries, Encyclopedias [also considered tertiary]     * Histories     * Journal articles [depending on the discipline, they can be primary]     * Magazine and newspaper articles [this distinction varies by discipline]     * Textbooks [also considered tertiary]     * Web site [also considered primary]

• << Previous: Primary Sources
• Next: Tiertiary Sources >>
• Last Updated: Apr 11, 2024 1:27 PM
• URL: https://libguides.usc.edu/writingguide

Root out friction in every digital experience, super-charge conversion rates, and optimize digital self-service

Uncover insights from any interaction, deliver AI-powered agent coaching, and reduce cost to serve

Increase revenue and loyalty with real-time insights and recommendations delivered to teams on the ground

Know how your people feel and empower managers to improve employee engagement, productivity, and retention

Take action in the moments that matter most along the employee journey and drive bottom line growth

Whatever they’re are saying, wherever they’re saying it, know exactly what’s going on with your people

Get faster, richer insights with qual and quant tools that make powerful market research available to everyone

Run concept tests, pricing studies, prototyping + more with fast, powerful studies designed by UX research experts

Track your brand performance 24/7 and act quickly to respond to opportunities and challenges in your market

Explore the platform powering Experience Management

• Free Account
• For Digital
• For Customer Care
• For Human Resources
• For Researchers
• Financial Services
• All Industries

Popular Use Cases

• Customer Experience
• Employee Experience
• Employee Exit Interviews
• Net Promoter Score
• Voice of Customer
• Customer Success Hub
• Product Documentation
• Training & Certification
• XM Institute
• Popular Resources
• Customer Stories

Market Research

• Artificial Intelligence
• Partnerships
• Marketplace

The annual gathering of the experience leaders at the world’s iconic brands building breakthrough business results, live in Salt Lake City.

• English/AU & NZ
• Español/Europa
• Español/América Latina
• Português Brasileiro
• REQUEST DEMO
• Experience Management
• Secondary Research

Try Qualtrics for free

Secondary research: definition, methods, & examples.

19 min read This ultimate guide to secondary research helps you understand changes in market trends, customers buying patterns and your competition using existing data sources.

In situations where you’re not involved in the data gathering process ( primary research ), you have to rely on existing information and data to arrive at specific research conclusions or outcomes. This approach is known as secondary research.

In this article, we’re going to explain what secondary research is, how it works, and share some examples of it in practice.

Free eBook: The ultimate guide to conducting market research

What is secondary research?

Secondary research, also known as desk research, is a research method that involves compiling existing data sourced from a variety of channels . This includes internal sources (e.g.in-house research) or, more commonly, external sources (such as government statistics, organizational bodies, and the internet).

Secondary research comes in several formats, such as published datasets, reports, and survey responses , and can also be sourced from websites, libraries, and museums.

The information is usually free — or available at a limited access cost — and gathered using surveys , telephone interviews, observation, face-to-face interviews, and more.

When using secondary research, researchers collect, verify, analyze and incorporate it to help them confirm research goals for the research period.

As well as the above, it can be used to review previous research into an area of interest. Researchers can look for patterns across data spanning several years and identify trends — or use it to verify early hypothesis statements and establish whether it’s worth continuing research into a prospective area.

How to conduct secondary research

There are five key steps to conducting secondary research effectively and efficiently:

1.    Identify and define the research topic

First, understand what you will be researching and define the topic by thinking about the research questions you want to be answered.

Ask yourself: What is the point of conducting this research? Then, ask: What do we want to achieve?

This may indicate an exploratory reason (why something happened) or confirm a hypothesis. The answers may indicate ideas that need primary or secondary research (or a combination) to investigate them.

2.    Find research and existing data sources

If secondary research is needed, think about where you might find the information. This helps you narrow down your secondary sources to those that help you answer your questions. What keywords do you need to use?

Which organizations are closely working on this topic already? Are there any competitors that you need to be aware of?

Create a list of the data sources, information, and people that could help you with your work.

3.    Begin searching and collecting the existing data

Now that you have the list of data sources, start accessing the data and collect the information into an organized system. This may mean you start setting up research journal accounts or making telephone calls to book meetings with third-party research teams to verify the details around data results.

As you search and access information, remember to check the data’s date, the credibility of the source, the relevance of the material to your research topic, and the methodology used by the third-party researchers. Start small and as you gain results, investigate further in the areas that help your research’s aims.

4.    Combine the data and compare the results

When you have your data in one place, you need to understand, filter, order, and combine it intelligently. Data may come in different formats where some data could be unusable, while other information may need to be deleted.

After this, you can start to look at different data sets to see what they tell you. You may find that you need to compare the same datasets over different periods for changes over time or compare different datasets to notice overlaps or trends. Ask yourself: What does this data mean to my research? Does it help or hinder my research?

5.    Analyze your data and explore further

In this last stage of the process, look at the information you have and ask yourself if this answers your original questions for your research. Are there any gaps? Do you understand the information you’ve found? If you feel there is more to cover, repeat the steps and delve deeper into the topic so that you can get all the information you need.

If secondary research can’t provide these answers, consider supplementing your results with data gained from primary research. As you explore further, add to your knowledge and update your findings. This will help you present clear, credible information.

Primary vs secondary research

Unlike secondary research, primary research involves creating data first-hand by directly working with interviewees, target users, or a target market. Primary research focuses on the method for carrying out research, asking questions, and collecting data using approaches such as:

• Interviews (panel, face-to-face or over the phone)
• Questionnaires or surveys
• Focus groups

Using these methods, researchers can get in-depth, targeted responses to questions, making results more accurate and specific to their research goals. However, it does take time to do and administer.

Unlike primary research, secondary research uses existing data, which also includes published results from primary research. Researchers summarize the existing research and use the results to support their research goals.

Both primary and secondary research have their places. Primary research can support the findings found through secondary research (and fill knowledge gaps), while secondary research can be a starting point for further primary research. Because of this, these research methods are often combined for optimal research results that are accurate at both the micro and macro level.

Sources of Secondary Research

There are two types of secondary research sources: internal and external. Internal data refers to in-house data that can be gathered from the researcher’s organization. External data refers to data published outside of and not owned by the researcher’s organization.

Internal data

Internal data is a good first port of call for insights and knowledge, as you may already have relevant information stored in your systems. Because you own this information — and it won’t be available to other researchers — it can give you a competitive edge . Examples of internal data include:

• Database information on sales history and business goal conversions
• Information from website applications and mobile site data
• Customer-generated data on product and service efficiency and use
• Previous research results or supplemental research areas
• Previous campaign results

External data

External data is useful when you: 1) need information on a new topic, 2) want to fill in gaps in your knowledge, or 3) want data that breaks down a population or market for trend and pattern analysis. Examples of external data include:

• Government, non-government agencies, and trade body statistics
• Company reports and research
• Competitor research
• Public library collections
• Textbooks and research journals
• Media stories in newspapers
• Online journals and research sites

Three examples of secondary research methods in action

How and why might you conduct secondary research? Let’s look at a few examples:

1.    Collecting factual information from the internet on a specific topic or market

There are plenty of sites that hold data for people to view and use in their research. For example, Google Scholar, ResearchGate, or Wiley Online Library all provide previous research on a particular topic. Researchers can create free accounts and use the search facilities to look into a topic by keyword, before following the instructions to download or export results for further analysis.

This can be useful for exploring a new market that your organization wants to consider entering. For instance, by viewing the U.S Census Bureau demographic data for that area, you can see what the demographics of your target audience are , and create compelling marketing campaigns accordingly.

2.    Finding out the views of your target audience on a particular topic

If you’re interested in seeing the historical views on a particular topic, for example, attitudes to women’s rights in the US, you can turn to secondary sources.

Textbooks, news articles, reviews, and journal entries can all provide qualitative reports and interviews covering how people discussed women’s rights. There may be multimedia elements like video or documented posters of propaganda showing biased language usage.

By gathering this information, synthesizing it, and evaluating the language, who created it and when it was shared, you can create a timeline of how a topic was discussed over time.

3.    When you want to know the latest thinking on a topic

Educational institutions, such as schools and colleges, create a lot of research-based reports on younger audiences or their academic specialisms. Dissertations from students also can be submitted to research journals, making these places useful places to see the latest insights from a new generation of academics.

Information can be requested — and sometimes academic institutions may want to collaborate and conduct research on your behalf. This can provide key primary data in areas that you want to research, as well as secondary data sources for your research.

Advantages of secondary research

There are several benefits of using secondary research, which we’ve outlined below:

• Easily and readily available data – There is an abundance of readily accessible data sources that have been pre-collected for use, in person at local libraries and online using the internet. This data is usually sorted by filters or can be exported into spreadsheet format, meaning that little technical expertise is needed to access and use the data.
• Faster research speeds – Since the data is already published and in the public arena, you don’t need to collect this information through primary research. This can make the research easier to do and faster, as you can get started with the data quickly.
• Low financial and time costs – Most secondary data sources can be accessed for free or at a small cost to the researcher, so the overall research costs are kept low. In addition, by saving on preliminary research, the time costs for the researcher are kept down as well.
• Secondary data can drive additional research actions – The insights gained can support future research activities (like conducting a follow-up survey or specifying future detailed research topics) or help add value to these activities.
• Secondary data can be useful pre-research insights – Secondary source data can provide pre-research insights and information on effects that can help resolve whether research should be conducted. It can also help highlight knowledge gaps, so subsequent research can consider this.
• Ability to scale up results – Secondary sources can include large datasets (like Census data results across several states) so research results can be scaled up quickly using large secondary data sources.

Disadvantages of secondary research

The disadvantages of secondary research are worth considering in advance of conducting research :

• Secondary research data can be out of date – Secondary sources can be updated regularly, but if you’re exploring the data between two updates, the data can be out of date. Researchers will need to consider whether the data available provides the right research coverage dates, so that insights are accurate and timely, or if the data needs to be updated. Also, fast-moving markets may find secondary data expires very quickly.
• Secondary research needs to be verified and interpreted – Where there’s a lot of data from one source, a researcher needs to review and analyze it. The data may need to be verified against other data sets or your hypotheses for accuracy and to ensure you’re using the right data for your research.
• The researcher has had no control over the secondary research – As the researcher has not been involved in the secondary research, invalid data can affect the results. It’s therefore vital that the methodology and controls are closely reviewed so that the data is collected in a systematic and error-free way.
• Secondary research data is not exclusive – As data sets are commonly available, there is no exclusivity and many researchers can use the same data. This can be problematic where researchers want to have exclusive rights over the research results and risk duplication of research in the future.

When do we conduct secondary research?

Now that you know the basics of secondary research, when do researchers normally conduct secondary research?

It’s often used at the beginning of research, when the researcher is trying to understand the current landscape . In addition, if the research area is new to the researcher, it can form crucial background context to help them understand what information exists already. This can plug knowledge gaps, supplement the researcher’s own learning or add to the research.

Secondary research can also be used in conjunction with primary research. Secondary research can become the formative research that helps pinpoint where further primary research is needed to find out specific information. It can also support or verify the findings from primary research.

You can use secondary research where high levels of control aren’t needed by the researcher, but a lot of knowledge on a topic is required from different angles.

Secondary research should not be used in place of primary research as both are very different and are used for various circumstances.

Questions to ask before conducting secondary research

Before you start your secondary research, ask yourself these questions:

• Is there similar internal data that we have created for a similar area in the past?

If your organization has past research, it’s best to review this work before starting a new project. The older work may provide you with the answers, and give you a starting dataset and context of how your organization approached the research before. However, be mindful that the work is probably out of date and view it with that note in mind. Read through and look for where this helps your research goals or where more work is needed.

• What am I trying to achieve with this research?

When you have clear goals, and understand what you need to achieve, you can look for the perfect type of secondary or primary research to support the aims. Different secondary research data will provide you with different information – for example, looking at news stories to tell you a breakdown of your market’s buying patterns won’t be as useful as internal or external data e-commerce and sales data sources.

• How credible will my research be?

If you are looking for credibility, you want to consider how accurate the research results will need to be, and if you can sacrifice credibility for speed by using secondary sources to get you started. Bear in mind which sources you choose — low-credibility data sites, like political party websites that are highly biased to favor their own party, would skew your results.

• What is the date of the secondary research?

When you’re looking to conduct research, you want the results to be as useful as possible , so using data that is 10 years old won’t be as accurate as using data that was created a year ago. Since a lot can change in a few years, note the date of your research and look for earlier data sets that can tell you a more recent picture of results. One caveat to this is using data collected over a long-term period for comparisons with earlier periods, which can tell you about the rate and direction of change.

• Can the data sources be verified? Does the information you have check out?

If you can’t verify the data by looking at the research methodology, speaking to the original team or cross-checking the facts with other research, it could be hard to be sure that the data is accurate. Think about whether you can use another source, or if it’s worth doing some supplementary primary research to replicate and verify results to help with this issue.

We created a front-to-back guide on conducting market research, The ultimate guide to conducting market research , so you can understand the research journey with confidence.

In it, you’ll learn more about:

• What effective market research looks like
• The use cases for market research
• The most important steps to conducting market research
• And how to take action on your research findings

Download the free guide for a clearer view on secondary research and other key research types for your business.

Related resources

Market intelligence 10 min read, marketing insights 11 min read, ethnographic research 11 min read, qualitative vs quantitative research 13 min read, qualitative research questions 11 min read, qualitative research design 12 min read, primary vs secondary research 14 min read, request demo.

Ready to learn more about Qualtrics?

• Architecture and Design
• Asian and Pacific Studies
• Business and Economics
• Classical and Ancient Near Eastern Studies
• Computer Sciences
• Cultural Studies
• Engineering
• General Interest
• Geosciences
• Industrial Chemistry
• Islamic and Middle Eastern Studies
• Jewish Studies
• Library and Information Science, Book Studies
• Life Sciences
• Linguistics and Semiotics
• Literary Studies
• Materials Sciences
• Mathematics
• Social Sciences
• Sports and Recreation
• Theology and Religion
• Publish your article
• The role of authors
• Promoting your article
• Abstracting & indexing
• Publishing Ethics
• Why publish with De Gruyter
• How to publish with De Gruyter
• Our book series
• Our subject areas
• Your digital product at De Gruyter
• Contribute to our reference works
• Product information
• Tools & resources
• Product Information
• Promotional Materials
• Orders and Inquiries
• FAQ for Library Suppliers and Book Sellers
• Repository Policy
• Free access policy
• Open Access agreements
• Database portals
• For Authors
• Customer service
• People + Culture
• Journal Management
• How to join us
• Working at De Gruyter
• Mission & Vision
• De Gruyter Foundation
• De Gruyter Ebound
• Our Responsibility
• Partner publishers

Your purchase has been completed. Your documents are now available to view.

A typology of secondary research in Applied Linguistics

Secondary research is burgeoning in the field of Applied Linguistics, taking the form of both narrative literature review and especially more systematic research synthesis. Clearly purposed and methodologically sound secondary research contributes to the field because it provides useful and reliable summaries in a given domain, facilitates research dialogues between sub-fields, and reduces redundancies in the published literature. It is important to understand that secondary research is an umbrella term that includes numerous types of literature review. In this commentary, we present a typology of 13 types of well-established and emergent types of secondary research in Applied Linguistics. Employing a four-dimensional analytical framework, focus, review process, structure, and representation of text of the 13 types of secondary research are discussed, supported by examples. This article ends with recommendations for conducting secondary research and calls for further inquiry into field-specific methodology of secondary research.

1 Secondary research in Applied Linguistics: commentary or research?

Secondary research, or literature review, refers to a scholarly review of a body of literature on a selected topic ( Ellis 2015 ). In the field of Applied Linguistics, and elsewhere in the social and natural sciences, there are two main types of secondary research: traditional and systematic ( Norris and Ortega 2007 ). Traditional (i.e., narrative-based) reviews bring together findings on the topic under investigation while providing a connection between the reported research and the macroscopic research terrain. While traditional reviews are often composed in a manner which aims to “tell a story” ( Norris and Ortega 2006 , p. 5), they also tend to include a critique of the literature at hand (an alternative name of some traditional reviews is “ critical reviews ”). This is especially true in traditional reviews which are published as stand-alone journal articles in which researchers identify strengths and weaknesses of existing studies on a topic. Unlike systematic reviews, traditional reviews do not generally extract data in any formalized way from primary studies. Finally, such reviews usually also make suggestions for future research directions based on the reviewers’ expert knowledge and/or based on gaps identified in the literature.

The other main family of secondary research is best characterized by a formal set of methods that are applied to the review process. Such ‘systematic reviews’, or research syntheses, have gained widespread popularity in Applied Linguistics research in recent years, especially those that aggregate quantitative findings ( Chong et al. 2023 ). In addition to systematic reviews, a growing body of papers concerning their methodology can also be found (e.g., Chong and Plonsky 2021 ; Chong and Reinders 2021 ; Chong and Reinders 2022 ; Li and Wang 2018 ; Macaro 2020 ; Norris and Ortega 2006 ; Plonsky and Brown 2015 ; Plonsky and Oswald 2015 ) and several new contributions are underway (e.g., Norris and Plonsky In Preparation ; Sterling and Plonsky In Preparation ). Indeed, Norris and Ortega’s (2007) prediction that research syntheses “will continue to thrive in our field” (p. 812) has been borne out. Unlike traditional or narrative reviews, systematic research syntheses refer to a “protocol-driven and quality-focused approach” ( Bearman et al. 2012 , p. 625) to aggregate research evidence to enlighten theory, research, policy, and practice.

In parallel to primary research, systematic research syntheses can be broadly divided into two major types: qualitative and quantitative . Qualitative research synthesis, also called “qualitative evidence synthesis” (e.g., Chong and Reinders 2020 ), assembles qualitative research evidence to “reveal deep insights into disparate literature for future research” ( Chen 2016 , p. 387). Another purpose of qualitative research synthesis is to strengthen and deepen qualitative evidence by unraveling “multidimensions, varieties, and complexities” ( Çiftçi and Savas 2017 , p. 4) amongst studies.

Quantitative research syntheses, as the name indicates, rely on quantitative data as a means to understand a given domain. The most well-known synthesis of this type is likely meta-analysis, which involves the statistical aggregation of effect sizes across studies (e.g., In’nami and Kozumi 2009 ). However, a number of other types of quantitative syntheses exist such as bibliometric reviews (see below). Figure 1 presents a visual of the basic breakdown of secondary research types introduced thus far.

Major types of secondary research.

Whilst traditional literature reviews continue to appear in Applied Linguistics journals and other outlets such as book chapters and monographs, systematic research syntheses are gaining prominence. Such growth can be attributed to several factors. First, the status of research synthesis has changed in recent years from a kind of ‘state-of-the-art’ think piece or commentary to a valuable form of empirical research necessary to consolidate and advance understanding of a given domain. To be sure, the systematicity, rigor, and transparency associated with research syntheses has greatly diminished the presence of bias and allowed for greater trust to be placed in secondary results. Secondly, reviews are no longer only produced by senior scholars who offer their authoritative opinions but also by more junior scholars as well as by synthesists working alongside researchers who specialize in the topic of the review (e.g., Visonà and Plonsky 2020 ). Such collaborations between experts in the substantive domain, on one hand, and in research syntheses on the other, bring a powerful synergy to the task of systematically reviewing a given body of work. A third reason for growth in the prominence of research synthesis is its place as “a cornerstone of the evidence-based practice and policy movement” ( Dixon-Woods et al. 2007 , p. 375) in education, medicine, and many other applied fields. Fourth, systematic research syntheses facilitate “epistemological diversity” ( Norris and Ortega 2007 , p. 813). Through more systematic and comprehensive literature searches (see Plonsky and Brown 2015 ), systematic research syntheses give voice to research findings published not only in prominent journals (mostly published in English) but also local and regional journals as well as other less visible yet valuable sources, such as masters and doctoral theses.

Despite the growth and prominence of secondary research, we have noticed a lack of consistency in naming conventions in the synthetic literature in applied linguistics as well as in other disciplines (e.g., Sutton et al. 2019 ). More importantly, underlying this inconsistency appears to be a misunderstanding of the different approaches to synthesis and their corresponding techniques and goals. One outcome of such inconsistency is confusion among authors, reviewers, editors, and readers ( Chong et al. 2022 ). Perhaps more serious is the implicit message sent to readers regarding this type of work that almost anything goes and by any name. Consequently, we see an urgent need to clarify not only the labels but the characteristics and accepted norms for conducting systematic research syntheses in Applied Linguistics.

2 A typology of secondary research in Applied Linguistics

In the previous section, we established that secondary research in Applied Linguistics has experienced a shift from the narrative tradition to a more systematic, empirical approach. While the development of systematic research synthesis is gaining momentum, it is unlikely that traditional literature reviews will be replaced entirely. Rather, in our view, it benefits the whole research field that the two families of secondary research co-exist, serving different purposes to advance our understanding. With this in mind, and to provide structure and guidance to future synthetic efforts, we address the following question in the remainder of this paper: “ What are the different types of secondary research in Applied Linguistics ?”

A number of taxonomies or typologies have been proposed to standardise conventions and practices of secondary research, for example, in healthcare ( Grant and Booth 2009 ), health sciences ( Littell 2018 ), medical sciences ( Munn et al. 2018 ), and social sciences ( Cooper 1988 ). With the proliferation of secondary research in Applied Linguistics in recent years, we believe the development of a field-specific typology is necessary. At the outset, however, a few disclaimers are warranted. Firstly, this typology is not a result of a comprehensive and fully systematic literature search, although considerations were given as to what reviews to include as examples in the typology (see below). Secondly, given the nature of this article (which is a commentary), analysis of the included reviews will indubitably represent, to a certain extent, our own views, biases, and experiences as primary and secondary/synthetic researchers. Thirdly, this typology is not to be seen as conclusive or definitive. Owing to the dynamic development of secondary research in Applied Linguistics, we regard this typology as a work in progress and invite contributions and revisions which build upon it. Finally, the purpose of presenting this preliminary typology is not to prescribe best practices in secondary research. Rather, we hope but to offer an overview of this versatile and emergent set of methodological techniques as a means to standardize some of the naming and methodological conventions being employed. We also seek to expose the field to some of the potential approaches to and outcomes of secondary research that might otherwise be unfamiliar or overlooked.

In the remainder of this article, we discuss features of 13 types of secondary research in Applied Linguistics. Discussions on relevant features are based on the frameworks by Gough et al. (2012) and Grant and Booth (2009) . The features analyzed include purpose of review, process of review (search strategy, appraisal mechanism, synthesis techniques, analysis techniques), and product of review (i.e., structure and representation of text). Features of each type of secondary research will first be discussed (see Table 1 ). Then, the 13 types of secondary research will be compared to identify issues and shared attributes (see breakdown in Table 2 ) and suggest a way forward. It must be emphasized that the types of literature review/secondary research above are by no means mutually exclusive. It is common, in fact, for secondary studies to overlap. For example, Tullock and Ortega (2017) is a scoping review and a meta-analysis.

A typology of research synthesis in applied linguistics.

A summary of the 13 types of secondary research in Applied Linguistics.

2.1. Overview of the 13 types of research synthesis in Applied Linguistics

Critical review: The purpose of a critical review is to explore prevalent views in a research topic and offer alternative perspectives. Muñoz and Singleton (2011) , for instance, reviews and challenges the prevalent view of maturational constraints toward second language acquisition. Moreover, this type of review often discusses key questions within the target domain. Lai and Li (2011) provide another example in their critical review on the intersection of technologies and task-based language teaching. Regarding the review process, there is generally no explicit description of how literature included in the review was searched and appraised, nor does it mention how the findings from existing research are synthesized and analyzed. The text of a critical review of literature takes a narrative approach, discussing prevalent issues thematically, with arguments or views illustrated by individual studies. In terms of product, a critical review often has a customized structure with headings related to the specific prevalent issues. Unlike the majority of secondary research types, information in a critical review is usually represented using only text and without the use of figures or tables.

Meta-analysis: Meta-analysis is one of the most prominent types of systematic research synthesis in Applied Linguistics ( Plonsky 2014 ). It aims to consolidate similarities and clarify conflicting findings. Given its quantitative nature, synthesists who conduct meta-analyses are often interested in effects or efficacy of a particular intervention. For example, Kang and Han’s (2015) meta-analysis quantified the overall effects of written corrective feedback on improving linguistic accuracy of second language writers; as is typical in meta-analysis, the study also identified a number of factors which moderate the effect of written corrective feedback. The review process of meta-analysis is documented in detail, usually in a separate ‘method’ section, with sub-sections dedicated to the literature search, coding process, and analysis. The two meta-analyses included in Table 1 reported how data were synthesized through coding of the features of primary studies and how the reliability of the coding was upheld (e.g., Teimouri et al. 2019 ). The synthesized data were then analyzed using descriptive and inferential statistics to estimate and identify effect sizes, outliers, publication bias, and moderators. The procedures involved in ‘reliability generalization meta-analysis’, a particular type of meta-analysis, are nearly identical to meta-analysis. However, the focus is on aggregating estimates of measurement error (see, e.g., Plonsky 2017 , 2019 ) rather than effect sizes. As a type of systematic research synthesis, meta-analysis follows a rather standard structure like a primary study, usually with additional materials included as supplementary online documents (e.g., coding scheme of Teimouri et al. 2019 , is made available on IRIS). In addition to text-based explanations, meta-analyses frequently utilise figures, tables, and bullet points to document the review procedure and make the otherwise information-dense piece more reader-friendly.

Methodological synthesis: This type of research synthesis can take two different approaches. Some focus on assessing the methodological approaches, designs, tools, and so forth within a given substantive domain, as in Plonsky and Gass’ (2011) review of interactionist research in second language acquisition (SLA). Other methodological syntheses take as their defining principle a particular research technique or tool. This approach can be seen, for example, in Marsden et al.’s (2018) review of self-paced reading tasks. Concerning the review process, the two reviews included in Table 1 represent a similarly systematic approach ( Crowther et al. 2021 ; Plonsky and Kim 2016 ). Similar to meta-analysis, Plonsky and Kim (2016) documented the coding process and how reliability of coding was maintained. In relation to how data were analyzed, the calculated percentages and frequencies of different research and reporting practices. These data are used to describe and evaluate the methods in the domain in question as well as to provide empirically-grounded recommendations for future research. Like other types of research synthesis, the structure of a methodological synthesis conforms to that of the typical empirical study in the social sciences.

Mixed review: As the name suggests, this type of research synthesis is a combination of two types of review. For example, Jackson and Suethanapornkul (2013) is a combination of a qualitative research synthesis and meta-analysis on task complexity whilst Marsden et al.’s (2018) extensive work is a presentation of a narrative and systematic literature review on replication studies in second language research. The review process, textual structure, and representation vary and are contingent on the types of studies being included. For instance, Jackson and Suethanapornkul (2013) follow a systematic approach to research synthesis with explicit description of how the included literature was searched, selected, extracted, and synthesized. As for structure, two approaches are noted. In Jackson and Suethanapornkul (2013) , a convergent approach was adopted by presenting the qualitative research synthesis and meta-analysis as one single review, with the findings of each complementing each other (like the convergent design of mixed-methods studies noted in Creswell and Clark 2017 ). Another structural approach is sequential , meaning that one review is presented at a time (e.g., a narrative review is first presented in Marsden et al. 2018 and then the systematic literature review). When there are two reviews in a given report (sequential), the first is usually broader with the second one being based on a subset of the first (e.g., Yan et al.’s 2015 , review and meta-analysis of elicited imitation). Because of the profound breadth and depth of the work (see, e.g., Marsden et al. 2018 ), mixed reviews often include additional online materials.

Narrative review: This research-focused review aims to map the ‘state of the art’ in a given domain. Often written by leaders in a particular domain, the objective of this type of review is “to authoritatively answer particular research questions, and to identify gaps in research methodologies” ( Norris and Ortega 2006 , p. 4). Referring to the two included narrative reviews in Table 1 , there seems to be divergence in terms of how the review process is enacted. In DeKeyser and Botana (2015) , a narrative review is presented in the form of a traditional research synthesis with a customised structure addressing various facets of the research topic of L2 grammar acquisition. On the contrary, Thomson and Derwing (2015) adopt a more systematic approach, which is exemplified from the brief section on how studies were appraised and how findings of the included studies were analysed by calculating percentages. (We would note, however, that Thomson and Derwing’s choice to include this information is atypical of this type of review.) Both reviews include individual studies as examples of points of interest to substantiate their arguments. However, neither of these narrative reviews describe their search strategy or synthesis procedure. It appears that both reviews adopt a thematic approach (as reflected in the section headings) despite not stating how these themes are generated. In terms of structure, sections similar to a typical primary study are found in Thomson and Derwing (2015) whilst a thematically-driven organisation is found in DeKeyser and Botana (2015) . Usually only textual representation of results is found in these narrative reviews.

Qualitative research synthesis: This type of systematic research synthesis is dedicated to the aggregation of qualitative research evidence, usually in classroom-based studies, to unravel complexities of ecological and naturalistic research studies. Chong and Reinders (2020) for example, synthesizes findings from students’ and teachers’ perception of technology-mediated task-based language teaching. Like other types of systematic research synthesis such as meta-analysis, qualitative research synthesis strictly follows a systematic procedure of searching and selecting relevant literature. Due to its qualitative nature, much attention is paid to ensure ‘openness’ in the data coding process to fully capture emerging themes. Both Chen (2016) and Chong and Reinders (2020) adopt the constant comparison method of grounded theory to perform initial, focused, and axial coding of data ( Charmaz 2006 ). Unlike meta-analysis, however, reliability of coding of qualitative research synthesis is usually achieved through multiple rounds of discussions between reviewers. As for its structure, qualitative research synthesis follows the standard structure of a research paper, with important textual information underscored using diagrams and tables.

Research agenda: The purpose of a research agenda is to “discuss research tasks” ( Svalberg 2012 , p. 376) and “review existing studies … to summarize current knowledge and to identify research gaps” ( Lou and Noels 2019 , p. 1). Similar to the aforementioned types of traditional literature reviews, authors of a research agenda do not specify the methods used to search and screen relevant studies. As far as analysis of literature is concerned, at least two forms can be employed. First, as illustrated in Svalberg (2012) , the reviewed literature is presented narratively and thematically. In each theme, the reviewer offers an overview statement or argument which is supplemented with a few studies as examples. Each section concludes with research tasks which spring from the limitations of the reviewed studies. Another approach is the one adopted in Lou and Noels (2019) which focuses on an emerging research field, mindsets in language teaching and learning. In their research agenda, the authors first offer a review of the current research base in a thematically threaded fashion, analogous to Svalberg’s (2012) approach. Nevertheless, instead of adding research tasks at the end of each thematic section, Lou and Noels (2019) examine the way forward in a separate conclusion section. Regardless of the approach, a research agenda is usually structured thematically with section headings. Sometimes, crucial information is presented in tabular form to enhance clarity and impact (e.g., the construct of engagement with language in Svalberg 2012 ).

Research into practice: The objective of this type of research synthesis is self-explanatory. Being a specific type of review of the journal Language Teaching , a ‘research into practice’ review resembles a narrative review but with a more practical focus, that is, the application of research findings in language classrooms. For instance, Lee (2013) and Graham (2017) examine aspects of research on written corrective feedback and listening strategies which are over-and under-applied in classrooms. As a type of traditional research synthesis, there is no mention of how the topical literature was collected and assessed. Taking a thematic approach to analyze literature, pedagogically-related topics are examined by citing individual studies, and in some cases, with the addition of personal experience (e.g., Graham 2017 ). A “research into practice” synthesis adopts a customized structure presented in textual form.

Scoping review: Taking a systematic and inclusive approach, a scoping review aims to map the research landscape of a given domain ( Pham et al. 2014 ). Adhering to its systematic nature, the review process is conducted in a structured manner and each step in the process is reported extensively to maintain transparency, objectivity, and credibility (e.g., the inclusion and exclusion criteria). The included research is coded, and measures are in place to ascertain inter-coder reliability (e.g., through coder training in Visonà and Plonsky 2020 ). As for the analysis of coded research, synthesists conducting a scoping review usually present the overview of a research base by calculating frequencies and percentages. Depth of analysis is maintained through the discussion of individual studies of interest ( Tullock and Ortega 2017 ). Due to their systematic nature, scoping reviews are structured as a standardized research paper, with texts being accompanied by tables and figures. Finally, although scoping reviews tend to appear as ‘stand-alone’ pieces, they can also serve to identify specific subdomains ripe for further inquiry via meta-analysis or systematic reviews.

State-of-the-art review: State-of-the-art reviews are usually a more exhaustive and comprehensive review of a specific research topic. For example, the journal Language Teaching requires authors of state-of-the-art reviews to include approximately 100 works in the bibliography ( Language Teaching 2020 ). Ultimately, the mission of this kind of research synthesis is to “point out contradictions and omissions – as well as – agreement” ( Language Teaching 2020 ) in the field. In the two examples, methods pertaining to the literature search and selection are explicitly mentioned in one ( Hanks 2019 ) but not the other ( Bardovi-Harlig and Stringer 2010 ). As for data analysis, a more personal approach is adopted to offer an insider’s perspective. For example, a dialogic approach between the author and other researchers is adopted in Hanks (2019) and a thematic approach is adopted in Bardovi-Harlig and Stringer (2010) . In both cases, the analysis of the literature is done through the provision of overview statements, followed by substantiation with a number of studies as examples. Generally being more extensive, a state-of-the-art review often relies on tables and figures in addition to narrative exposition (see Bardovi-Harlig and Comajoan-Colomé 2020 for an example, which is published as a dedicated article type called “State of the Scholarship” in Studies in Second Language Acquisition ).

Systematic literature review: A systematic literature review (also called ‘research synthesis’) can be practice-focused or research-focused. Practice-oriented systematic literature reviews provide arguments in favor of a practice) and evaluate issues pertaining to enactment of such practice ( Macaro et al. 2018 ). As for research-focused systematic literature reviews, their aim is to assess the quality and range of studies and provide an updated literature search to inform new research questions ( Macaro et al. 2018 ). Literature search strategies are devised and the selection of literature is carefully planned and executed. The included literature is coded by multiple reviewers and the reliability of coding is emphasised. ‘Data’ are analyzed using thematic analysis and, like other types of systematic reviews, by calculating frequencies and percentages to observe patterns and trends in the sample. In many ways, a systematic literature review can be seen as overlapping with meta-analysis in all respects except for the aggregation of effect sizes. A typical research paper structure is adopted, with texts supplemented with figures and tables.

Historical review: The review types discussed thus far generally treat their target domains as a whole or perhaps in different groupings based on features or variables shared across certain studies. By contrast, historical reviews use time as an organizing principle and as a means to understand the theoretical and/or methodological development that has taken place in a given domain. Boo et al. (2015) , for example, traced the recent history of L2 motivation research, focusing on changes in theoretical models, research designs, and target languages, among other features. The authors describe their methods in detail and followed procedures much like those of other systematic review types, presenting results in tables and figures to enhance clarity. Historical reviews can also, however, take the form of a timeline. The journal Language Teaching has published many reviews of this type. Timelines generally begin with a broad introduction to situate the domain in question and to highlight major themes and/or developments that have taken place. Landmark publications and other pertinent events are then presented chronologically, usually accompanied by annotations as well as thematic indicators presented in the introduction. See Isaacs and Harding’s (2017) timeline of pronunciation assessment in L2 research as a prime example of this type of synthesis.

Bibliometric review: As in other types of reviews, bibliometric analyses focus on the study/report as the unit of analysis. However, the focus here is on publication meta-data such as citation counts (co-)citation within and across publications, authorship attributes (e.g., number, gender, language and geographic background), article titles and keywords, and so forth. Bibliometric research is fairly new to Applied Linguistics, but it appears to be surging as evident in the number of recent studies in a bibliography of bibliometric research in applied linguistics (Plonsky, N.D). Hyland and Jiang (2019) , for example, analyzed the frequency as well as linguistic and stylistic features of citation patterns both over time and across four disciplines including applied linguistics. (For another recent bibliometric analysis of citation patterns, see Lei and Liu 2018 .) Aryadoust and Ang (2021) take on a narrower domain, that of eye-tracking research in the language sciences. Using a sample of 341 publications, the authors extract and analyze citations and co-citations as well as research affiliations and countries, among other types of publication meta-data. The authors’ co-citation analysis found a number of prominent ‘clusters’ of scholarly activity and influence. As in other quantitative and systematic review types, bibliometric analyses often utilize tables and figures to present their findings.

3 Four continua of research synthesis in Applied Linguistics

Having laid out the 13 types of secondary research being employed in Applied Linguistics, we now turn to four dimensions, or continua, that characterize and distinguish between them. These include (a) research versus practice; (b) more versus less systematic; (c) more versus less structured presentation; and (d) multi-versus mono-modal (see Figure 2 ). We note that new continua and ways of describing secondary research many be needed as new types of secondary research emerge.

Four areas for analyzing secondary research in applied linguistics.

3.1 The ‘research-versus practice-focused’ continuum

Unsurprisingly, the objective of the majority of secondary research types is research-oriented. Purely research-focused reviews can be used to summarize the current state of research or aggregate research evidence (e.g., state-of-the-art review), to explore emerging research areas (e.g., scoping review), to challenge mainstream perspectives (e.g., critical review), to address methodological issues in research (e.g., methodological synthesis), or to suggest new research directions (e.g., research agenda). On the other hand, two types of secondary research, systematic literature review and qualitative research synthesis, can address both research-and practice-related issues. For instance, in their qualitative research synthesis, Chong and Reinders (2020) draw implications from the synthesized research evidence to inform technology-mediated task-based language teaching research and practice. As for systematic literature reviews address and express caution around the use of course grades as metrics to assess language proficiency levels in classrooms and in L2 research. Amongst the 13 types of secondary research identified previously, there is only one type which focuses exclusively on pedagogical practices: research into practice. ‘Research into practice’ reviews concern the extent to which findings from classroom-based studies are applicable to authentic educational contexts. In particular, reviewers are interested in underscoring areas of research findings which are under-, over-, and possibly mis-applied.

There is an unequal focus of secondary work in Applied Linguistics on research and researchers as the sole interest and audience, respectively, as opposed to teaching and teachers, for example. Currently, there is a dearth of reviews which focus on practice, as exemplified from the fact that ‘research into practice’ reviews can only be found with regularity in one journal (i.e.,  Language Teaching ). While it is the primary purpose of most secondary research to target researchers, it is meaningful to explore review types which focus primarily on informing the practice of language teaching and learning as well as other practical realms within applied linguistics such as assessment ( Chong 2020 ). In addition to evaluating the success of knowledge transfer from academia to the professional context, practice-focused secondary research can focus on synthesizing data from naturalistic, classroom-based studies to showcase evidence-based practices. Moreover, it is important for synthesists to specify the purpose of their reviews. The connection between the purported goal of the reviews and the review process should be stated explicitly. ‘Meta-reviews’, or ‘reviews of reviews’ can be conducted to examine the alignment between the objectives in each type of secondary research. Moreover, investigations can be carried out on the (mis)alignment between the purpose of research syntheses and their review processes.

3.2 The ‘more systematic versus less systematic’ continuum

Regarding systematicity of the review process, the reviewed examples show that there is a fairly even distribution of the 13 types of research synthesis on the continuum. Five types of secondary research adopt a systematic approach and traditional (non-empirical) approach to reviewing, respectively (meta-analysis, systematic literature review, qualitative research synthesis, scoping review, methodological review). Moreover, three types (mixed review, narrative review, state-of-the-art review) utilize a mixture of systematic and narrative review strategies. Research syntheses which embrace the more systematic review process include a discrete ‘methodology’ section detailing the various stages of the review process namely literature search strategy, inclusion and exclusion criteria, a coding scheme for data extraction/collection, and data analysis. To reduce bias and increase credibility, these systematic research syntheses are transparent with their coding procedure, often sharing their coding scheme as supplementary materials or appendices (e.g., Chen 2016 ), calculating inter-coder reliability (e.g., Larsson et al. 2020 ), and describing strategies to resolve disagreement in the synthesis and analysis stage (e.g., resolving disagreement through discussions in Chong and Reinders 2020 ; Liu and Chong 2023 , in this special issue). By contrast, the three types of traditional secondary research (critical review, research into practice, research agenda), tend to employ a less systematic process. Resembling the ‘commentary’ style of secondary research, these research syntheses do not describe how the review process is conducted nor do they provide justifications for the adoption of a particular review protocol.

Based on the typology in Table 1 , another observation is the ambiguity of the review processes of some types of secondary research. With regards to the aforementioned examples, mixed reviews, narrative review, and state-of-the-art review do not demonstrate a consistent take on the systematicity of the process. While we can understand that the systematicity of mixed reviews is contingent on the mixture of the review types, it raises concerns regarding the stark contrast in the methods used in the paired examples of the other two review types. Without a consensual view towards review methodology, we are likely to encounter challenges for researchers who want to conduct such kinds of review. It is imperative to conduct comprehensive reviews on the methodologies used in each type of secondary research and the variation thereof as a means to bring consistency to the field. It would be fruitful to review methodologies of (a) earlier and more recent secondary research; (b) secondary research published in top-tiered and less prestigious journals/outlets; (c) written by researchers at various career stages, and (d) published in different sub-domains of Applied Linguistics. Despite being a strenuous task, such work would yield valuable insights into the differentiation between ‘ideal’, ‘acceptable’, and ‘unacceptable’ review protocols of each type of secondary research. (For an example of an evaluation of meta-analytic techniques in applied linguistics, see Plonsky and Talip 2015 ).

Secondly, three types of secondary research adopt a less systematic review process (critical review, research agenda, research into practice), relying mainly on the authors’ expertise and judgement to identify important themes and studies as examples. We recommend a dose of caution when interpreting such reviews because less systematicity can lead to inconsistency, idiosyncrasy, and bias (not that systematicity eliminates bias). The majority of these reviews do not include an explanation of how themes are generated or how studies are chosen. Although credibility of these reviews may not be an issue when they are written by leading scholars in the field who offer an ‘authoritative stance’, it is important to minimize bias by providing a brief statement on the reflective process of the reviewers. Hanks (2019) is a commendable example in this respect. This review, which adopts a “dialogic approach”, makes clear that the review process is rigorous despite not having “a third-party stance” (p. 145). The author acknowledges a personal stance and explains how reviewer perspectives are co-constructed through interactions with researchers, professionals, and doctoral students.

3.3 The ‘more standardized structure versus less standardized structure’ continuum

From a genre analysis perspective, some secondary research types adhere to a more standardized structure while structures of others are more flexible. The kinds of secondary research which follow a more standardized reporting structure are also those which implement a more systematic review protocol, including meta-analysis, systematic literature review, qualitative research synthesis, methodological synthesis, bibliometric review, and scoping review. Their standardized organizational approach comprises six structural components: introduction, background literature, method, findings, discussion, implications/conclusion, in parallel to a typical primary research paper. These types of research synthesis often include additional online supplementary information such as coding schemes and bibliographical summaries of the synthesized studies.

Secondary research types which adopt a less standardized structure are those which employ a less systematic and rigorous review procedure (i.e., critical review, research into practice, research agenda). These types of secondary research, which narrate the synthesized findings through a thematic approach, organize the review text based on prominent themes. An interesting observation emanating from this comparison is found in the two examples of narrative review. Traditionally, a narrative review is viewed as a type of secondary research that relies on a less systematic synthesis process; nevertheless, one narrative review mentioned previously follows a primary research paper structure ( Thomson and Derwing 2015 ), whilst the other uses a customized thematic structure ( DeKeyser and Botana 2015 ). This example, which demonstrates a lack of consensus of a preferred structure for narrative reviews, is made even more intriguing as these two reviews were published in the same issue of the same journal (i.e.,  Applied Linguistics ). While the discussion of only two examples makes it impossible to draw any definitive conclusion, it suggests that the two sets of authors of these review articles may have different perceptions or conceptualizations towards ‘narrative review’, with Thomson and Derwing (2015) leaning more towards the “synthesis as research” side while DeKeyser and Botana (2015) regard narrative review as an authoritative commentary.

The major takeaway, pertaining to the structural organization of secondary research, is quite apparent: Reviews that adopt a more systematic and scientific review protocol are represented in a more standardized structure while those that are written in a ‘commentary’ style are more flexible with their structures. From the examples drawn from the typology, the only type of secondary research that shows disagreement in terms of structural approach is narrative review. To further understand structural variations of each type of secondary research, genre analysis can be conducted to identify the “cognitive structuring” ( Garzone 2015 , p. 6) or “schematic structure of the discourse” ( Swales 1990 , p. 58) used to convey communicative purposes in each research synthesis type.

3.4 The ‘multimodal text versus monomodal text’ continuum

Referring to the analytical framework (see Figure 2 ), the representation of the review text is another dimension of variation across review types. Over half of the secondary research types adopt multimodal representation, using a combination of text, figures, and tables. Among these types of secondary research, six (meta-analysis, systematic literature review, qualitative research synthesis, scoping review, methodological synthesis, bibliometric review) adopt systematic review protocols. To present large sums of information in a clear and systematic manner, these reviews utilize tables and figures to document the steps of the review and collate bibliographical, substantive, and methodological information from the selected studies (e.g., Chen 2016 ). By contrast, the four types of text which generally rely on monomodal text (i.e., narrative review, critical review, research into practice, research agenda) represent the types of secondary research that follow a more flexible mechanism of review.

Similar to the previous continuum, the decision on whether to utilize different modes to present secondary results seems to be associated with the systematicity of the review. For more systematic types of reviews, figures and tables are usually used to present methodological flow and underscore synthesized findings. Bullet points are sometimes used to list factual information (e.g., the list of journals consulted, the list of search terms used, inclusion/exclusion criteria). Future research on secondary research practices might look into the “science of using science”, which refers to the study of “the efficacy of interventions applied to increase decision-makers’ use of research in various decision arenas” ( Laurenz et al. 2016 , p. 1). One form of intervention to enhance evidence-informed decision making that can be investigated is the mode of presentation of synthesized research evidence (e.g., the use of infographical representation of synthesized findings to facilitate use of research findings by practitioners).

4 Conclusion

This article provided an overview of the common types of secondary research in the field of Applied Linguistics (see Table 2 for a summary). The main objective of this piece is to demonstrate the similarities and differences among types of secondary research through four perspectives: topic (research/practice focus), review process (systematic/traditional), structure (standardized/customized), and representation of text (textual/multimodal). The typology presented will be useful to not only novice reviewers as a primer for what is possible using secondary methods, but also seasoned synthesists to explore different secondary research orientations and options. Most importantly, we hope that this typology can serve as a catalyst to facilitate discussions on secondary and synthetic research methodologies and, ultimately, lead to the formulation of methodological guidelines for each type of secondary research for our own field.

Aryadoust, Vahid & Bee H. Ang. 2021. Exploring the frontiers of eye tracking research in language studies: A novel co-citation scientometric review. Computer Assisted Language Learning 34(7). 898–933. https://doi.org/10.1080/09588221.2019.1647251 . Search in Google Scholar

Bardovi-Harlig, Kathleen & Llorenç Comajoan-Colomé. 2020. The aspect hypotheses and the acquisition of L2 past morphology in the last 20 years. Studies in Second Language Acquisition . 1–31. 10.1017/S0272263120000194 Search in Google Scholar

Bardovi-Harlig, Kathleen & David Stringer. 2010. Variables in second language attrition: Advancing the state of the art. Studies in Second Language Acquisition 32. 1–45. https://doi.org/10.1017/s0272263109990246 . Search in Google Scholar

Bearman, Margaret, Calvin D. Smith, Angela Carbone, Susan Slade, Chi Baik, Marnie Hughes-Warrington & David L. Neumann. 2012. Systematic review methodology in higher education. Higher Education Research and Development 31(5). 625–640. https://doi.org/10.1080/07294360.2012.702735 . Search in Google Scholar

Boo, Zann, Zoltan Dörnyei & Stephen Ryan. 2015. L2 motivation research 2005e2014: Understanding a publication surge and a changing landscape. System 55. 145–157. https://doi.org/10.1016/j.system.2015.10.006 . Search in Google Scholar

Brown, Alan V, Luke Plonsky & Yasser Teimouri. 2020. The use of course grades as metrics in L2 research: A systematic review. Foreign Language Annals 51(4). 763–778. 10.1111/flan.12370 Search in Google Scholar

Charmaz, Kathy. 2006. Constructing grounded theory , 2nd edn. Thousand Oaks, CA: Sage. Search in Google Scholar

Chen, Tsuiping. 2016. Technology-supported peer feedback in ESL/EFL writing classes: A research synthesis. Computer Assisted Language Learning 29(2). 365–397. https://doi.org/10.1080/09588221.2014.960942 . Search in Google Scholar

Chong, Sin Wang. 2020. The role of research synthesis in facilitating research-pedagogy dialogue. ELT Journal 74(4). 484–487. https://doi.org/10.1093/elt/ccaa046 . Search in Google Scholar

Chong, Sin Wang, Tinjun Lin & Yulu Chen. 2022. A methodological review of systematic literature reviews in higher education: Heterogeneity and homogeneity. Educational Research Review 35. 100426. https://doi.org/10.1016/j.edurev.2021.100426 . Search in Google Scholar

Chong, Sin Wang & Luke Plonsky. 2021. A primer on qualitative research synthesis in TESOL. Tesol Quarterly 55(3). 1024–1034. https://doi.org/10.1002/tesq.3030 . Search in Google Scholar

Chong, Sin Wang & Hayo Reinders. 2020. Technology-mediated task-based language teaching: A qualitative research synthesis. Language, Learning and Technology 24(3). 70–86. https://doi.org/10125/44739 . Search in Google Scholar

Chong, Sin Wang & Hayo Reinders. 2021. A methodological review of qualitative research synthesis in CALL: The state-of-the-art. System 103. 102646. https://doi.org/10.1016/j.system.2021.102646 . Search in Google Scholar

Chong, Sin Wang & Hayo Reinders. 2022. Autonomy of English language learners: A scoping review of research and practice. Language Teaching research . Advanced online publication. https://doi.org/10.1177/13621688221075812 . Search in Google Scholar

Chong, Sin Wang, Bond, Melissa & Hamish Chalmers. 2023. Opening the methodological black box of research synthesis in language education: where are we now and where are we heading? Applied Linguistics Review . https://doi.org/10.1515/applirev-2022-0193 . Search in Google Scholar

Çiftçi, Emrullah Y. & Perihan Savas. 2017. The role of telecollaboration in language and intercultural learning: A synthesis of studies published between 2010 and 2015. ReCALL 30(3). 278–298. https://doi.org/10.1017/s0958344017000313 . Search in Google Scholar

Cooper, Harris M. 1988. Organizing knowledge synthesis: A taxonomy of literature reviews. Knowledge in Society 1. 104–126. https://doi.org/10.1007/bf03177550 . Search in Google Scholar

Creswell, John W. & Vicki L. P. Clark. 2017. Designing and conducting mixed methods research , 3rd edn. Thousand Oaks, CA: Sage. Search in Google Scholar

Crowther, Dustin, Susie Kim, Jongbong Lee, Jungmin Lim & Shawn Loewen. 2021. Methodological synthesis of cluster analysis in second language research. Language Learning 71. 99–130. https://doi.org/10.1111/lang.12428 . Search in Google Scholar

DeKeyser, Robert & Goretti P. Botana. 2015. The effectiveness of processing instruction in L2 grammar acquisition: A narrative review. Applied Linguistics 36(3). 290–305. https://doi.org/10.1093/applin/amu071 . Search in Google Scholar

Dixon-Woods, Mary, Alex Sutton, Rachel Shaw, Tina Miller, Jonathan Smith, Bridget Young, Sheila Bonas, Andrew Booth & David Jones. 2007. Appraising qualitative research for inclusion in systematic reviews: A quantitative and qualitative comparison of three methods. Journal of Health Services Research and Policy 12(1). 42–47. https://doi.org/10.1258/135581907779497486 . Search in Google Scholar

Ellis, Rod. 2015. Introduction: Complementarity in research syntheses. Applied Linguistics 36(3). 285–289. https://doi.org/10.1093/applin/amv015 . Search in Google Scholar

Garzone, Giuliana E. 2015. Genre analysis. In Karen Tracy, Cornelia Ilie & Todd Sandel (eds.), The international encyclopaedia of language and social interaction . Hoboken, NJ: Wiley. 10.1002/9781118611463.wbielsi069 Search in Google Scholar

Gough, David, James Thomas & Sandy Oliver. 2012. Clarifying differences between review designs and methods. Systematic Reviews 1. 28. https://doi.org/10.1186/2046-4053-1-28 . Search in Google Scholar

Graham, Suzanne. 2017. Research into practice: Listening strategies in an instructed classroom setting. Language Teaching 50(1). 107–119. https://doi.org/10.1017/s0261444816000306 . Search in Google Scholar

Grant, Maria J. & Andrew Booth. 2009. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal 26. 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x . Search in Google Scholar

Hanks, Judith. 2019. From research-as-practice to exploratory practice-as-research in language teaching and beyond. Language Teaching 52. 143–187. https://doi.org/10.1017/s0261444819000016 . Search in Google Scholar

Hyland, Ken & Feng Jiang. 2019. Points of reference: Changing patterns of academic citation. Applied Linguistics 40. 64–85. https://doi.org/10.1093/applin/amx012 . Search in Google Scholar

In’nami, Yo & Rie Koizumi. 2009. A meta-analysis of test format effects on reading and listening test performance: Focus on multiple-choice and open-ended formats. Language Testing 26(2). 219–244. https://doi.org/10.1177/0265532208101006 . Search in Google Scholar

Isaacs, Talia & Luke Harding. 2017. Pronunciation assessment. Language Teaching 50. 347–366. https://doi.org/10.1017/s0261444817000118 . Search in Google Scholar

Jackson, Daniel O. & Sakol Suethanoapornkul. 2013. The cognition hypothesis: A synthesis and meta-analysis of research on second language task complexity. Language Learning 63(2). 330–367. https://doi.org/10.1111/lang.12008 . Search in Google Scholar

Kang, EunYoung & Zhaohong Han. 2015. The efficacy of written corrective feedback in improving L2 written accuracy: A meta-analysis. The Modern Language Journal 99(1). 1–18. https://doi.org/10.1111/modl.12189 . Search in Google Scholar

Lai, Chun & Guofang Li. 2011. Technology and task-based language teaching: A critical review. CALICO Journal 28(2). 498–521. https://doi.org/10.11139/cj.28.2.498-521 . Search in Google Scholar

Language Teaching . 2020. Instructions for authors : Cambridge University Press. Available at: https://www.cambridge.org/core/journals/language-teaching/information/instructions-contributors . Search in Google Scholar

Larsson, Tove, Magali Paquot & Luke Plonsky. 2020. Inter-rater reliability in learner corpus research: Insights from a collaborative study on adverb placement. International Journal of Learner Corpus Research 6. 237–151. https://doi.org/10.1075/ijlcr.20001.lar . Search in Google Scholar

Laurenz, Laurenz, Janice Tripney & David Gough. 2016. The Science of using science: Researching the Use of research Evidence in decision-making : University College London. Available at: https://www.alliance4usefulevidence.org/assets/Science-of-Using-Science-Final-Report-2016.pdf . Search in Google Scholar

Lee, Icy. 2013. Research into practice: Written corrective feedback. Language Teaching 46(1). 108–119. https://doi.org/10.1017/s0261444812000390 . Search in Google Scholar

Lei, Lei & Dilin Liu. 2018. Research trends in applied linguistics from 2005 to 2016: A bibliometric analysis and its implications. Applied Linguistics 40. 540–561. https://doi.org/10.1093/applin/amy003 . Search in Google Scholar

Li, Shaofeng & Hong Wang. 2018. Traditional literature review and research synthesis. In Aek Phakiti, Peter De Costa, Luke Plonsky & Sue Starfield (eds.), Palgrave handbook of applied linguistics research methodology : Palgrave. 10.1057/978-1-137-59900-1_6 Search in Google Scholar

Littell, Julia H. 2018. Conceptual and practical classification of research reviews and other evidence synthesis products. Campbell Systematic Reviews 14(1). 1–21. https://doi.org/10.4073/cmdp.2018.1 . Search in Google Scholar

Lou, Nigel M. & Kimberly A. Noels. 2019. Promoting growth in foreign and second language education: A research agenda for mindsets in language learning and teaching. System 86. 10212. https://doi.org/10.1016/j.system.2019.102126 . Search in Google Scholar

Liu, Qi & Sin Wang Chong. 2023. Bilingual education in China: A qualitative synthesis of research on models and perceptions. Applied Linguistics Review . 10.1515/applirev-2022-0194 Search in Google Scholar

Macaro, Ernesto, Samantha Curle, Jack Pun, Jiangshan An & Julie Dearden. 2018. A systematic review of English medium instruction in higher education. Language Teaching 51(1). 36–76. https://doi.org/10.1017/s0261444817000350 . Search in Google Scholar

Macaro, Ernesto 2020. Systematic reviews in applied linguistics. In Jim McKinley & Heath Rose (eds.), The Routledge handbook of research methods in applied linguistics : Routledge. 10.4324/9780367824471-20 Search in Google Scholar

Marsden, Emma, Kara Morgan-Short, Sophie Thompson & David Abugaber. 2018. Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning 68(2). 321–391. https://doi.org/10.1111/lang.12286 . Search in Google Scholar

Marsden, Emma, Sophie Thompson & Luke Plonsky. 2018. A methodological synthesis of self-paced reading in second language research. Applied PsychoLinguistics 39. 861–904. https://doi.org/10.1017/S0142716418000036 . Search in Google Scholar

Munn, Zachary, Cindy Stern, Edoardo Aromataris, Craig Lockwood & Zoe Jordan. 2018. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Medical Research Methodology 18. 1–9. https://doi.org/10.1186/s12874-017-0468-4 . Search in Google Scholar

Muñoz, Carmen & David Singleton. 2011. A critical review of age-related research on L2 ultimate attainment. Language Teaching 44(1). 1–35. https://doi.org/10.1017/s0261444810000327 . Search in Google Scholar

Norris, John M. & Lourdes Ortega. 2006. Synthesizing research on language learning and teaching . Amsterdam, The Netherlands: John Benjamins Publishing Company. 10.1075/lllt.13 Search in Google Scholar

Norris, John M. & Lourdes Ortega. 2007. The future of research synthesis in applied linguistics: Beyond art or science. Tesol Quarterly 41(4). 805–815. https://doi.org/10.1002/j.1545-7249.2007.tb00105.x . Search in Google Scholar

Norris, John M. & Luke Plonsky. In preparation. Research synthesis and meta-analysis in applied linguistics: A practical guide . New York, NY: Routledge. Search in Google Scholar

Pham, Mai T., Andrijana Rajić, Judy D. Greig, Jan M. Sargeant, Andrew Papadopoulos & Scott A. McEwen. 2014. A scoping review of scoping reviews: Advancing the approach and enhancing consistency. Research Synthesis Methods 5(4). 371–385. https://doi.org/10.1002/jrsm.1123 . Search in Google Scholar

Plonsky, Luke. 2014. Study quality in quantitative L2 research (1990-2010): A methodological synthesis and call for reform. The Modern Language Journal 98. 450–470. https://doi.org/10.1111/j.1540-4781.2014.12058.x . Search in Google Scholar

Plonsky, Luke. 2017. Quantitative research methods in instructed SLA. In Shawn Loewen & Masatoshi Sato (eds.), The Routledge handbook of instructed second language acquisition , 505–521. New York, NY: Routledge. 10.4324/9781315676968-28 Search in Google Scholar

Plonsky, Luke. 2019. Language learning strategy instruction: Recent research and future directions. In Anna U. Chamot & Vee Harris (eds.), Learning strategy instruction in the language classroom: Issues and implementation , 3–21. Bristol, UK: Multilingual Matters. 10.21832/9781788923415-007 Search in Google Scholar

Plonsky, Luke & Dan Brown. 2015. Domain definition and search techniques in meta-analyses of L2 research (Or why 18 meta-analyses of feedback have different results). Second Language Research 31. 267–278. https://doi.org/10.1177/0267658314536436 . Search in Google Scholar

Plonsky, Luke & Susan Gass. 2011. Quantitative research methods, study quality, and outcomes: The case of interaction research. Language Learning 61. 325–366. https://doi.org/10.1111/j.1467-9922.2011.00640.x . Search in Google Scholar

Plonsky, Luke & YouJin Kim. 2016. Task-based learner production: A substantive and methodological review. Annual Review of Applied Linguistics 36. 73–97. https://doi.org/10.1017/S0267190516000015 . Search in Google Scholar

Plonsky, Luke & Frederick L. Oswald. 2015. Meta-analyzing second language research. In Luke Plonsky (ed.), Advancing quantitative methods in second language research , 106–128. New York, NY: Routledge. 10.4324/9781315870908-6 Search in Google Scholar

Plonsky, Luke & Talip Gönülal. 2015. Methodological synthesis in quantitative L2 research: A review of reviews and a case study of exploratory factor analysis. Language Learning 65(Supp. 1). 9–36 (edited by John M. Norris, Steven, J. & Rob Schoonen). https://doi.org/10.1111/lang.12111 . Search in Google Scholar

Sterling, Scott & Luke Plonsky. In preparation. Meta-research in applied linguistics . New York, NY: Routledge. Search in Google Scholar

Sutton, Anthea, Mark Clowes, Louise Preston & Andrew Booth. 2019. Meeting the review family: Exploring review types and associated information retrieval requirements. Health Information and Libraries Journal 36(3). 202–222. https://doi.org/10.1111/hir.12276 . Search in Google Scholar

Swales, John. 1990. Genre analysis . Cambridge, UK: Cambridge University Press. Search in Google Scholar

Svalberg, Agneta M.-L. 2012. Language awareness in language learning and teaching: A research agenda. Language Teaching 45(3). 376–388. https://doi.org/10.1017/s0261444812000079 . Search in Google Scholar

Teimouri, Yasser, Julia Goetze & Luke Plonsky. 2019. Second language anxiety and achievement: A meta-analysis. Studies in Second Language Acquisition 41. 363–387. https://doi.org/10.1017/S0272263118000311 . Search in Google Scholar

Thomson, Ron I. & Tracey M. Derwing. 2015. The effectiveness of L2 pronunciation instruction: A narrative review. Applied Linguistics 36(3). 326–344. https://doi.org/10.1093/applin/amu076 . Search in Google Scholar

Tullock, Brandon & Lourdes Ortega. 2017. Fluency and multilingualism in study abroad: Lessons from a scoping review. System 71. 7–21. https://doi.org/10.1016/j.system.2017.09.019 . Search in Google Scholar

Visonà, Mark W. & Luke Plonsky. 2020. Arabic as a heritage language: A scoping review. International Journal of Bilingualism 24. 559–615. https://doi.org/10.1177/1367006919849110 . Search in Google Scholar

Yan, Xun, Yukiko Maeda, Jing Lv & April Ginther. 2015. Elicited imitation as a measure of second language proficiency: A narrative review and meta-analysis. Language Testing 33(4). 497–528. https://doi.org/10.1177/0265532215594643 . Search in Google Scholar

© 2023 the author(s), published by De Gruyter, Berlin/Boston

This work is licensed under the Creative Commons Attribution 4.0 International License.

• X / Twitter

Supplementary Materials

Please login or register with De Gruyter to order this product.

Systematic Literature Review: Inter-Reletedness of Innovation, Resilience and Sustainability - Major, Emerging Themes and Future Research Directions

• Review paper
• Published: 25 July 2022
• Volume 3 , pages 1157–1185, ( 2023 )

Cite this article

• N. Zupancic   ORCID: orcid.org/0000-0002-5582-8687 1  

3543 Accesses

Explore all metrics

Research has been using resilience, sustainability and innovation interchangeably, but there is a lack of research that would provide an insight into how they are related to each other. This systematic literature review thus investigates research on sustainability, innovation and resilience, how they are related to each other, and also identifies major, emerging themes and future research directions on these topics.

We used Bibliometrix software to visually describe articles with the highest number of citations, to present the thematic evolution of the field and present a historical map. The triangulation and thematic groups were identified and compared by two independent researchers. 

Resilience is involved in processes, sustainability is concerned with the outcomes, while innovation represents a pathway to achieving both resilience and sustainability. Resilience can ensure the provision of the system functions in the face of shocks and stresses and sustainability can ensure the adequate performance of the system in general. Three major themes were identified, ‘socio-ecological systems’, ‘transformational innovation’ and ‘political governance’, as well as three emerging themes, ‘food security and agriculture’, ‘businesses and finance’ and ‘interconnected systems’. There is a need for longitudinal, multi-scale and interdisciplinary research that would explore various aspects of integrating these concepts. 

There is a great overlap between the concepts of resilience, sustainability and innovation. Future research could study these concepts in relation to each other. 

Similar content being viewed by others

Three pillars of sustainability: in search of conceptual origins

Ben Purvis, Yong Mao & Darren Robinson

The costs and benefits of environmental sustainability

Paul Ekins & Dimitri Zenghelis

Leverage points for sustainability transformation

David J. Abson, Joern Fischer, … Daniel J. Lang

Avoid common mistakes on your manuscript.

Introduction

Countries and societies face issues as a result of climate change, such as excessive or insufficient precipitation, rising sea levels, extreme temperature changes, storms, droughts, floods and other climate hazards that are only going to increase in the future [ 72 , 100 ], thus making it ever more important to prepare for [ 40 ]. Resilience enables adaptivity to an unknown future [ 75 ], and sustainability, as defined by the United Nations Brundtland Commission [ 92 ], is concerned about meeting the needs of the present without compromising the ability of the future generations to meet their own needs. Resilience is important in overcoming the inevitable problems that arise when faced with unpredicted shocks and stressors [ 75 ]. The two concepts can be regarded as vital if humanity is to successfully face the major changes that may occur in the near future [ 70 ]. However, the distinguishment among both is still debated and differently approached by the authors [ 70 , 76 ]. On the other hand, the underlying concept that could help societies to achieve both resilience and sustainability can be found in innovation. Leach et al. [ 40 ] define innovation as new ways of doing things, both in science and technology, but also associated with institutions and social practices.

Innovative approaches are needed to deal with large-scale changes [ 22 , 97 ], steer away from potential Earth system thresholds [ 77 ] and to build the resilience of social-ecological systems, so they are better able to deal with changes as opportunities [ 97 , 98 ]. Innovation has so far often occurred without reference to the issue of ecological integrity, even though it is essential to consider this before implementation as many innovations can have considerable ecological and societal risks (Olsson et al. [ 70 ]). However, we must also consider technological advances that have the potential to combat climate change. What was once considered as science fiction is now slowly moving into the centre of international climate change discussions, research and politics [ 21 ]. Such innovations can be closely linked with the concepts of sustainability and resilience. In fact, the overlap between resilience and sustainability is considerable, as can also be seen by Lebel et al. [ 41 ], suggesting that the critical factors for sustainability are resilience, the capacity to cope and adapt, and the conservation of sources of innovation and renewal. Therefore, one could argue that we are talking about very similar concepts, which are inter-related. In this study, we systematically review the academic literature and examine the empirical examples of how sustainability, resilience and innovation have been applied simultaneously to understand the interrelations among the concepts and their application in diverse settings. While much research has been performed to study sustainability, resilience and innovation, there is a significant lack of research that would assess the relations among them in a systematic way. Our study thus aims to overcome this gap and to identify how the concepts are interlinked and to provide a framework to study these concepts in practice.

RQ1: How are the concepts of sustainability, resilience and innovation related?

Many authors suggest that an interdisciplinary approach is needed to study sustainability, resilience and innovation [ 13 , 14 ], with a multitude of stakeholders involved in the process [ 12 ]. For future research studies that aim to integrate these three concepts, it is therefore necessary to understand how they have been studied empirically and where gaps in the literature may occur. The implementation of sustainability, resilience and innovation requires a multi-scale and multi-stakeholder approach [ 40 ]. We are therefore interested to understand what research methods were used and at what scales were they studied at. In this context, Lebel et al. ([ 41 ], pp.1) furthermore argue that we should ask not only The resilience of what, to what? but also For whom? Continued involvement from a variety of stakeholders is integral to effective decision-making and institutionalisation of programmes in the long run [ 6 ]. In further research or in applying these concepts in practice, researchers may also need to better understand the variety of stakeholders involved in the process, the research methods used to study such concepts empirically and at what scales are they are referred to.

RQ2: What research methods are used by the researchers to study resilience, innovation and sustainability?

RQ3: What scales authors referred to in the study of innovation, resilience and sustainability?

RQ4: Who are the stakeholders involved in the study of resilience, sustainability and innovation?

Our paper provides a review of the most influential publications, as well as newly published articles, to identify the major trends within this research and what the most promising directions are for new research studies. We use a systematic literature review, which has a high level of evidence, as represented by the evidence-based pyramid [ 85 ]. A bibliographic mapping was used for a visual interpretation of the connections between groups. Most articles reviewed were published in journals such as Ecosystems , Ecology , International Journal of Agricultural Sustainability , Journal of Environmental Planning and Management , Local Environment , Ecology and Systems , AMBIO , PNAS , Climate Risk Management and Global Environmental . Two independent researchers triangulated the themes to identify major and emerging themes among the highly cited and newly published articles. The related research has adequately addressed all three concepts, and our review of the literature found 269 articles that refer to resilience, sustainability and innovation simultaneously. However, no research was found that would address all three concepts in a systematic way. Our review of the literature identifies major and emerging themes of research, which can provide guidelines for future research directions in the field of sustainability, resilience and innovation.

RQ5: What are the major and emerging themes within the field of sustainability, resilience and innovation?

RQ6: What are the future research directions on innovation, resilience and sustainability?

Our systematic review of literature maps out the data from Thomson Reuters’ Web of Science, where we searched for all the articles referring to resilience, sustainability and innovation simultaneously. The rest of this article proceeds as follows: In ‘ Research Methodology ’, we discuss the methodology used in developing the article; ‘ Results ’ presents the results, using descriptive statistics to map out the authors with the highest number of citations and a historical mapping approach to identify and discuss the three major themes referring to ‘political governance’, ‘socio-ecological systems’ and ‘transformational innovation’. In the second part of ‘ Results ’, we identify the emerging themes of ‘food security and agriculture’, ‘business and finance’ and ‘interconnected systems’. In ‘ Discussion ’, we present a discussion of the findings of our research, give an overview of the inter-relatedness among the fields of sustainability, resilience and innovation and provide a framework to study these concepts in practice. We then provide a critical overview of the major and emerging themes. Based on our findings, we provide new research directions in ‘ Future Research Directions ’. ‘ Conclusion ’ then concludes the article and sets an agenda for further research on sustainability, resilience and innovation.

Research Methodology

Our study used a method of systematic literature review. Systematic literature reviews differ from traditional narrative reviews since they enable researchers to adopt replicable, scientific and transparent processes of analysing research. By using technology to carry out a systematic literature review, we were able to overcome some of the normal bias in literature searches [ 49 ]. The methodology includes a set of applied procedures and techniques, which provide an insight into information about the specific topic of research, which supports the overall scientific credibility of the study. To visually describe the articles included in the review, we have used the Bibiloshiny package [ 11 ] within the Bibliometrix command in R, which allowed us to import from the bibliographic database in the Web of Science. The bibliographic mapping approach is commonly used to systematically review a field of research and its influential publications. The result of this method is a bibliographic map that enables researchers to visually describe the structure of the literature in a chronological manner, providing information on the most cited works and the relationships among them [ 49 ]. We used this approach as it enabled us to develop an objective assessment of the topics of interest [ 45 , 48 ].

Data Collection and Data Cleaning

The study was pre-registered on the OSF platform [ 106 ]. Our article follows the steps in data collection, as outlined by Janssen et al. [ 29 ], Janssen [ 28 ], Linnenluecke et al. [ 48 ], and Linnenluecke et al. [ 46 ]. In the Thomson Reuters Web of Science platform, we searched for articles referring to ‘sustainability’, ‘resilience’ and ‘innovation’. Studies referring to all three concepts were included in our review. The initial search included 376 records. Following the example of previous researchers Linnenluecke et al. [ 49 ], Janssen et al. [ 29 ], Janssen [ 28 ] and Linnenluecke et al. [ 46 ], the records were manually cleaned by two reviewers that checked the title, abstract and keywords for each record. For each article, we retrieved the following information: name(s) of the author(s), title of the article, name of the journal, citation details (volume, issue and page numbers), as well as abstract and keywords. After the initial review, articles referring to urban planning, peace engineering, cultural heritage and historical development were excluded from further review. The final literature review included 269 articles. We have included studies in the review that have the largest number of citations for major themes, which were listed on the historical map. Finally, we review all the studies that were published up to one year before the review took place.

Data Analysis and Synthesis

We visually represented the descriptive analysis of the most influential articles by using a thematic evolution technique and a historical bibliographic mapping approach. Two researchers independently assessed and identified major themes based on the bibliographic map and then triangulated the research topics by reading the abstracts or articles in detail, if necessary. The researchers independently triangulated research studies into groups and then compared the results. Based on discussions, they agreed on the major themes of research. A similar procedure was followed to identify emerging themes, just without the visual representation. The researchers looked through those articles published up to one year before the data was collected to identify emerging themes. They then independently classified them into groups based on the similarities of research topics. Both times, two independent researchers classified the research into groups independently and then, after discussions, decided on the major and emerging themes. When the groups were identified, the analysis of the research groups followed. Based on the major and emerging themes, new directions for future work were identified for each of these.

Descriptive Statistics

The author with the most citations in the review was Olsson, followed by Folke, Galaz and Thompson. We followed the example of Linnenluecke [ 47 ] in using a cut-off score set at the point where the local citation score (LCS) was levelling off, which happened to be at 6 or more citations, which showed 36 authors among those with the most local citations (Fig.  1 ).

Number of citations per leading authors

Thematic Evolution

Looking at the thematic evolution, we can see how research evolved over time (Fig.  2 ). The general trend shows the fragmentation of research into sub-topics from 1999–2018 to 2019–2021. In 1999–2018, we see corporate social responsibility, management, dynamics, resilience and knowledge as the major themes of research. In 2019–2021, we see the main topics of have become systems, transition, sustainability, impact, adaptation, knowledge, policy, vulnerability and agriculture. Resilience was the largest field of research by far from 1999 to 2018 and then fragmented into various sub-topics referring to sustainability, impact, adaptation, knowledge, policy and vulnerability. What is interesting is that resilience is not a topic of such interest in the later years, from 2019 to 2021, while sustainability only starts to become the topic of interest from 2019 to 2021, when the highest number of articles were published on the sustainability domain. The theme referring to management has in recent years transitioned towards systems, sustainability and impact, while the theme referring to science has fragmented to more specific topics, such as sustainability, adaptation and knowledge. Knowledge is a much smaller topic of interest in 2019–2021 than in 1999–2018, and it has fragmented into agriculture, policy and knowledge.

Thematic evolution

Historical Mapping and Major Themes of Research

Historical mapping was used to identify the major thematic groups. Even though the historical mapping does not include every possible article included in this review, it has enabled us to arrive at a classification of major research streams, which were identified by manually looking through the articles and categorising them into their research groups. In Table 1 , top publications are listed with a local citation score (LCS), which is the number of citations by authors included in the study, and a global citation score (GCS), which is the number of citations that the publication has received in total. Most articles with the highest number of citations were published in 2014, although several were published in later years (Fig.  3 ). The predominant approaches among the highly cited articles were a literature review (nine out of 19 articles), six case studies and three example reviews, one used a qualitative method and one a global quantitative study. Three major thematic groups of articles were identified, encompassing social-ecological systems, transformational innovation and political governance.

Major themes of research among highly cited articles

Transformational Innovation

The first group of research works was concerned with how transformational innovation can be reached at the level of systems or society and how it can contribute towards achieving resilience and sustainability. Olsson et al. [ 70 , 71 ] and Westley et al. [ 98 ] highlighted the importance of linking innovation to ecological integrity, with the main issue being how innovation is used [ 98 ]. Innovation within research has represented a tool to ensure resilience and achieve sustainability [ 60 , 62 , 69 , 98 ]. Furthermore, innovation represents a tool for the transformation of socio-ecological systems [ 63 , 70 , 71 , 97 ]. Olsson et al. [ 70 ] argue that scholars should make a distinction between adaptation and transformation when referring to resilience. Transformation must necessarily be included when looking at resilience in relation to the concept of sustainability [ 63 ]. Two types of innovation are identified as those that have transformative capacities to achieve a large-scale transformation: social and technological innovation. In the domain of technological innovation, Olsson et al. [ 70 ] referred to information and communication technology, nano- and biotechnology, synthetic biology, new energy systems and geo-engineering techniques. Within social innovation, they referred to new modes of governance, business models, microcredits and crowd sourcing. Social innovation is a more frequent theme in contributing to resilience and sustainability [ 57 , 60 , 62 , 63 , 70 , 97 , 98 ] than technological innovation [ 70 , 98 ]. Two case studies were identified that studied how the processes of social innovation can have effects on ecosystem-based land-use planning, namely Oak Ridges Moraine [ 57 ] and Great Barrier Reef Marine Park [ 69 ].

Both top-down and bottom-up approaches can be considered as sources of innovation [ 98 ]. Moore et al. [ 60 , 62 ] studied social innovation through a global fellowship programme and showed that introducing the insights of individual fellows included in the programme into new complex system dynamics can contribute towards transformational innovation. However, Moore et al. [ 63 ] argued that while social movements create conditions towards transformability, they are not the transformation itself. Disruption only creates opportunities for change to occur within the system. Individuals can interpret the problem and mobilise others to self-organise around a new idea or practice that addresses the issue [ 63 ]. Transformational innovations often require altering the dominant power structures and embedding the newly reconfigured social-ecological elements and feedback within institutions. The new trajectory itself then gains power, which is why a multi-stakeholder approach is necessary to adequately address transformational innovation [ 60 , 62 ]. Research regarding innovation should encompass both the related power structures and various stakeholders when implementing resilience [ 70 ]. Frameworks to study the implementation of transformational innovation have been provided by many authors (Hölscher et al. 2019, [ 60 , 62 , 63 , 97 , 98 ]) while two groups of researchers used the adaptive cycle as a theory of how systems can implement innovation to ensure resilience [ 57 , 97 ].

Change demands innovation across multiple scales [ 98 ], as a transformation can occur on a single scale and lead to change on multiple scales [ 63 , 70 ]. Researchers who consider the issue of transformability within that of resilience note that the scale dimension is necessary, as this will indicate whether factors should be considered in contrast or even conflict [ 70 ]. Westley et al. [ 98 ] consider macro-, meso- and micro-scales as those that need to be considered if transformational innovation can achieve long-term change. At a macro-scale, authors refer to the political, economic, cultural and legal institutions that should be transformed away from what favours environmental destruction. At the meso-scale, authors refer to the problem or domain scale, which represents the opportunities that need to be incorporated to promote novelty and innovation. Finally, the micro-scale refers to individuals and small groups where invention originates and where the early sources of disruptive or catalytic innovation can be found. Various authors [ 69 , 70 , 97 ] often refer to the size of the transformation on a scale dimension (large vs. small scale). However, while Olsson et al. [ 69 ] focus on large-scale innovation in order to achieve a large-scale change, small niche innovations have also shown the capacity to scale up and transform institutions [ 63 , 97 , 98 ].

Research has considered a wide variety of stakeholders when referring to resilience, sustainability and innovation, ranging from social, economic and state actors [ 60 , 62 , 70 , 71 , 98 ] to environmental organisations such as park authorities, scientific and policy agencies, as well as the broader community. Moore et al. [ 63 ] were less specific in identifying stakeholders, as they consider either individuals or collectives. According to Olsson et al. [ 71 ], individual stakeholders included in their study were entrepreneurs and consumers. Westley et al. [ 97 ] refer to individuals or institutions. Besides individuals, McCarthy et al. [ 57 ] delineate collectives onto groups, organisations and agencies. Westley et al. [ 98 ] also consider NGOs and private sector firms as stakeholders.

Political Governance

Within the major theme of political governance, researchers have provided specific frameworks to understand how political decision-making can be guided to achieve adaptive co-management [ 6 , 7 ] and sustainability [ 40 ]. Butler et al. [ 6 ], Butler et al. [ 7 ] applied resilience theory to adaptation pathways, which can provide a decision-making framework whose aim is to push societies towards a more sustainable future by considering complex systems, uncertainty and multiple stakeholders. The use of an adaptive pathways approach, however, showed limited evidence for institutional change to existing processes [ 7 ].

Chapin et al. ([ 12 ], pp. 16,641) define resilience as the ‘capacity of social-ecological system to absorb shocks or perturbations and still retain fundamental function, structure, identity and feedbacks due to changing conditions’. Climate change works in interaction with population growth and ecosystem loss, which reduces land, water and food supply. Bad resilience may occur as a trap, which means that self-reinforcing social and ecological feedback can make moving to new trajectories extremely difficult. Poverty can remain due to corruption, traditional institutions and fatalism. Transformation is required when the system is trapped in an undesired state. Resilience can be improved by addressing the negative feedback that prevents systems from changing, thus fostering ecological, cultural, institutional and economic diversity, as well as adaptability [ 12 ]. Innovation can help the system to stay in its current state or to transform it [ 6 ], and it can foster sustainable development and resilience [ 40 ]. Adaptability can be enhanced through policies that promote learning and innovation [ 12 ]. A case study by Chapin et al. [ 12 ] of Alaskan boreal forests provided researchers with broad policy strategies that can be widely applicable in other settings. Socially innovative initiatives at the community level can, through the co-ordinating role of the state, be scaled up and act as a pressure for more participatory forms of governance when implementing grassroots innovation [ 1 ]. Grassroots innovations, in this way, emerge as networks generating innovative solutions for climate change adaptation and mitigation and therefore work as strategies towards achieving sustainability and resilience [ 18 , 19 ].

Feola and Nunes’ [ 18 , 19 ] study is the only study on the major themes which acknowledged both local and global contextual factors in a case study of transition movements. Grassroots innovations support the process of local niche innovation creation and the incubation of socio-technical innovation. Less successful transition initiatives underestimate the importance of contextual factors and material resources in influencing success [ 18 , 19 ]. Sustainable development goals enable major transformations to take place, not only in policies and technologies but also in modes of innovation too. Leach et al. [ 40 ] examine examples from East Africa and Latin America and proposed a 3-D framework where the policymakers evaluate their decision-making based on direction, diversity and distribution when implementing innovation in moving towards sustainable development. Political governance can play an important role in designing, implementing as well as regulating an innovation. An innovation should address socio-ecological feedback and support the stewardship of ecosystem services [ 21 ]. Galaz [ 21 ] argue that geo-engineering and Earth stewardship are not necessarily in conflict, but rather can be viewed as complementary, which is why it is important to have institutional settings and regulations that are strong enough to prevent ecological risk while allowing for novelty, fail-safe experimentation and continuous learning.

When studying the diverse nature of political governance, we need to understand it within the cross-scale and multi-scale contexts [ 6 ,  7 ,  40 ]. Researchers have thus used various scales of reference, with the one referring to the spatial (large vs. small scale) being the predominant frame of reference [ 6 , 7 ,  12 , 21 , 40 ] followed by a territorial scale that considers local vs. global level of initiatives [ 6 , 12 , 18 , 19 , 40 ]. Some authors also consider related issues using a temporal scale and the notion of time [ 18 , 19 , 12 ]. Research in the major theme of political governance refers to a diverse set of stakeholders, with most of the studies including governmental institutions. Butler et al. [ 6 ,  7 ] classified stakeholders into individuals, institutions and governmental organisations. Leach et al. [ 40 ] referred to stakeholders as farmers and consumers. Feola and Nunes [ 18 , 19 ] have looked into how resilience, sustainability and innovation can be achieved from an NGO perspective, while Chapin et al. [ 12 ] are more specific and include resource harvesting, conservation, hazard reduction and ecological externality institutions as the topics of interest.

Socio-ecological Systems

The final group on major themes included articles concerned predominantly with the interactions among social-ecological systems. However, studying both the ecological as well as social dimensions has often been neglected by research studies [ 1 ]. Within this major theme, a substantial overlap can be found among the constructs of resilience, innovation and sustainability, while all concepts seem to be concerned about social and ecological interaction [ 1 , 13 , 14 ]. Cumming et al. [ 13 ] are the most influential authors within this theme. According to them, resilience is defined in terms of the system’s ability to maintain an identity while being faced with internal changes and external shocks and disturbances. The two aspects of identity can refer to the ecosystem (e.g. amount of focal habitat) and social factors (e.g. cultural groups). The relationship component within the definition refers to how the components fit together, while the continuity component represents the variables that maintain a system’s identity through space and time [ 14 ]. Resilience thinking in the integration of sustainability, resilience and innovation integrates both social and ecological aspects. Resilience focuses on the process of enhancing the likelihood that the system will be able to weather shocks or pass through inevitable adaptive cycles in an unproblematic manner, reduce serious vulnerabilities and move system states from an undesirable to a desirable state [ 13 ]. Cumming et al. [ 13 ] identified an exploratory framework for operationalisation and a measure of resilience which can be used for empirical studies across cases. This allows for the socio-ecological system to be studied, as well as to make predictions about whether properties of interest are resilient.

Progress towards sustainability in this theme depends on our understanding of socio-ecological systems. Resilience in the domain of socio-ecological systems is based on the interdisciplinary synthesis, which when applied to farming can lead towards sustainability [ 14 ]. Innovation can represent a part of or a pathway to achieving resilience [ 13 , 14 ]. Innovation in socio-ecological systems is more concerned with how variables that are related to novel solutions and responses to change can be shaped by biodiversity, cultural and livelihood diversity [ 13 ], as well as how learning and innovation can shape socio-ecological system interaction in fostering the resilience [ 14 ]. Darnhofer et al. [ 14 ] have shown that farming systems are too complex and variable over time and space for resilience models to give specific and closely predictive guidance to farmers. For a farm to achieve sustainability, it must take advantage of current opportunities while managing conditions that expand future possibilities, ensuring adaptability and transformability [ 14 ]. As such, sustainability considers the resilience of the system using both the short to the long term. Research work in socio-ecological systems builds on the idea of the adaptive cycles proposed by Holling [ 26 ], which consist of four phases: exploitation, conservation, release and reorganisation [ 13 , 14 ].

Interest in socio-ecological system interactions has remained on the temporal and spatial scales [ 1 , 14 ], with Baker and Mehmood [ 1 ] adding the functional scale. Each subsystem interacts with other subsystems at other spatial scales and other domains, which in turn are influenced by them. This is difficult to measure and thus make accurate predictions of how a system will respond, which gives rise to a need to address both the spatial and temporal mismatches that exist between a certain biophysical system and the governance system that is responsible for managing human–environment interactions. This mismatch may also occur when implementing social innovation, which can cause governance practices to fail with regard to promoting resilience on the social and ecological levels [ 1 ]. The research into socio-ecological systems looked at a diverse set of stakeholders ranging from public, private and civil society actors [ 1 ]. Stakeholders among them consider state agencies, universities, NGOs and communities [ 13 ], as well as farmers, local farmer groups associations and communities [ 14 ].

Emerging Research Trends

Bibliographic mapping investigates research that is well cited, and it does not consider newer research, which is why in the following section, we review the most recently published papers (up to a year before data collection started) and identify the most important themes in the most recent works. We review the papers that were published in one year before data collection. The results show that literature reviews [ 8 , 16 , 25 , 35 , 43 , 51 ,  55 , 67 , 79 ,  91 ] and case studies [ 2 , 4 , 9 ,  17 , 24 , 30 , 54 , 61 , 83 , 88 ] are still the predominant research methods, although more sophisticated methods of research are starting to become adopted, though, such as systematic literature reviews [ 15 ], comparative analysis of the investment criteria and other reports [ 33 ], interviews [ 23 , 81 ] and surveys [ 32 , 50 , 66 , 90 ].

Food Security and Agriculture

The first emerging theme, on food security and agriculture, encompasses the largest set of articles ([ 2 , 8 , 16 , 33 ,  55 , 67 , 76 , 79 , 81 , 83 , 91 ] which could be due to the rising need to consider food security within the frame of resilience, sustainability and innovation. The constraint of the natural resources should not exceed natural regenerative capacity, while the economic return should meet certain expectations to be considered sustainable [ 16 ]. Dong [ 16 ] argues that there is a paradigm shift occurring from efficiency-driven industrial agriculture to resilience-focused eco-friendly agriculture.

Sustainable systems require adequate performance across economic, social and environmental domains [ 76 , 83 ]. On the other hand, Reidsma et al. ([ 76 ], pp. 19) define resilience as the ‘ability to ensure the provision of the system functions in the face of complex and accumulating economic, social, environmental and institutional shocks and stresses’. It can thus be argued that there is a substantial overlap between resilience and sustainability. Resilience can work as a catalyst in sustaining the sustainability of the system when faced with external pressures and shocks, while sustainability is the umbrella that requires adequate performance in general and is therefore a broader concept and less specific. Resilience considers adaptability – ‘the capacity to actively respond to shock and stresses without changing the systems structures and feedback mechanisms’ – resilience also encompasses transformability, the ‘system’s capacity to reorganise its structure and feedback mechanisms in response to shocks and stresses’. While Reidsma et al. ([ 76 ], pp. 19) add the robustness of the system, which refers to ‘the capacity to resist and endure shocks or stressors’ ([ 76 pp. 19). When looking into the definition of resilience, a distinction was made by Reidsma et al. [ 76 ] between specific resilience, which considers what, to what and for what purpose, and general resilience, which considers the system’s overall robustness, adaptability and transformability. Studying these from the farming systems perspective, it was found that the resilience of the focal systems was perceived as low to moderate, and robustness and adaptability were often greater than transformability. Liu et al. [ 50 ] investigate technological innovation as the key to improving productivity in food production and agriculture. The implementation of technological innovation is linked to financial prosperity. Even in the newer research on food security and agriculture, the theory of adaptive cycles continues to be applied [ 5 ], while a new theory of change is presented by Seghieri et al. [ 83 ].

There is a need to define agroecological or sustainable innovations in the context of people [ 83 ]. Consumers have become a topic of interest. Sustainable agriculture can deliver benefits for human health, as well as prevent environmental sustainability from being compromised. However, each consumer is only a limited agent of change, which is why broader perspectives need to be considered within the farming and agriculture domain [ 81 ]. Researchers consider both the social domain regarding using the agriculture to reduce hunger and the related impacts on the ecological aspects of food system security, as the race to meet sustainable development goals to achieve zero hunger by 2030 increases the need to intensify agricultural production, which raises concerns for the related environmental footprint. The deployment of new and improved technologies, especially advanced biotechnology, can help reach the targets set [ 67 ].

In terms of food security and agriculture, the stakeholders involved in research on sustainability, resilience and innovation were farm owners and farming communities. However, different stakeholders were also involved in the agricultural value chain, such as governments, scientific communities, the public [ 16 ], non-governmental organisations and academic experts [ 81 ]. Research in this domain also acknowledges environmental parks [ 55 ], and primary forest owners [ 79 ] as those where resilience, sustainability and innovation play an important role. A broad array of stakeholders can be classified into public, civil and private stakeholders [ 83 ]. Benitez et al. [ 2 ] investigated how more specifically how females as stakeholders through participation and leadership contribute to positive economic, ecological and sociocultural changes in farming households and communities. Researchers looked at different scales, such as various spatial scales (small, medium vs. large) and territorial scale (municipal, provincial and national vs. local, national and international) [ 8 ,  16 , 33 ,  67 , 76 , 79 , 81 ], and individual vs. household scale [ 2 ], or at the level of the forest, farm, community and territory ([ 79 , 83 , 91 ]. The related technologies have the potential to play a role in improving the sustainability and resilience of food systems, at cell, plant, field and farm scales [ 8 ]. The notion of a temporal scale in terms of time has also been introduced into the discussion on food security and agriculture [ 81 , 91 ].

Business and Finance

The idea of a win–win strategy emerges in the business and finance domain. Organisations act as a catalyst for the sustainable development of society, while at the same time need to develop in a sustainable way to realise their potential. The concept of organisational resilience is complex, and methods of effectively modelling resilient organisations are still developing. Nyaupane et al. [ 66 ] performed a study at the Bureau of Land Management, the largest public land management agency in the USA, which investigated organisational resilience. The employee-organisation relationship (E-O-R) framework was developed to understand the relationship between employees’ skillsets, organisational traits and organisational resilience. It is possible to have employees whose skillset is adaptive, but it can occur that at the organisational level, such adaptability can be low, and thus, there are low resilience levels [ 66 ]. On the other hand, a survey of 455 organisations showed that long-term organisational resilience enables sustainable competitiveness through fostering dynamic capabilities [ 32 ]. Therefore, the concepts of resilience and sustainability are also interlinked in the business and finance domain.

Del Giudice et al. [ 15 ] further link the business model innovation theory and resilience theory within their research. Including socio-ecological systems, resilience theory and socio-technical transitions, the concept of transformational adaptation has been introduced to the study of climate change. Kasdan et al. [ 33 ] consider how transformational adaptation has influenced the funding priorities and financing of projects. They performed an analysis of investment criteria, board meetings, minutes, documents and reports under the convention of United Nations Framework Convention on Climate Change. Transformative potential guides funding decisions; however, it is important to consider whether transformational change is achievable and desirable at all times. Research also investigated how industry 4.0 adoption contributes towards sustainable business practice, and how digital technologies can improve resilience aftershocks [ 43 ], as well as how technology can be implemented towards achieving sustainability and resilience in the tourism sector [ 9 , 51 , 96 , 102 ], seaports [ 15 ], and barriers to implementing the innovation for pottery businesses in Iran [ 23 ]. During the COVID-19 pandemic, the resilience of global logistics has become an important topic in global supply chain management [ 3 ,  24 , 90 ]). Information sharing, logistics, networking and transportation are the most powerful factors that impact sustainable business and supply chain performance, while the need to move from corporate social responsibility towards also sustainability arises in this context [ 90 ]. Technological advances could help through the use of new technologies and the adoption of advanced analytics such as blockchain [ 3 ]. Research also investigates how innovation presents an important factor in implementing a circular economy [ 9 , 61 ].

Carraresi and Broring [ 9 ] argue that while research can often focus on innovation in a company’s business, an innovative solution does not lie only in a particular object or process, but in a frame of mind. Therefore, the behavioural aspects are starting to be considered as success factors in the implementation of innovations in business settings. The multilevel nature of an innovation is considered in business research, where the innovative output of nations is impacted at the local and organisational levels [ 23 ]. Stakeholders represent organisations [ 32 ], employees [ 66 ], customers [ 9 ] and governmental organisations [ 23 ]. Most researchers within the domain of business and finance do not refer to the scale used,only Moore et al. [ 61 ] refer to the national scale and Trivellas et al. [ 90 ] to the small scale.

Interconnected Systems

We can only understand the concepts of sustainability, resilience and innovation when we understand the complexity of the system we are considering. Kok et al. [ 35 ] acknowledge the importance of human agency, as well as non-human factors, such as technology. There are three important, mutually related dynamic properties of complex adaptive systems: emergence, self-organisation and adaptation. Within non-linear interactions, systematic feedback loops can emerge in the system, which allows small changes to either accelerate the systematic change through a positive feedback loop or diminish the systemic change through a negative feedback loop. This level of operationalisation proposes that we shift the analysis from the individual parts to the systems perspectives. The power dynamics agency has become a topic of interest in the transition towards sustainability, while individual-level factors such as the autonomy of actors and collaboration between them have also started to be topics of interest [ 35 ].

Sustainability is defined with seven attributes referring to productivity, stability, reliability, adaptability, equity, self-management and resilience [ 52 ]. Resilience is therefore considered as a part of sustainability. Nevertheless, they argue that sustainability cannot be measured directly, but must be measured through transversal evaluation by comparing management systems at the same time, or through longitudinal evaluation, by looking into the evolution of the system over time [ 54 ]. Sustainability can work in both ways – governance can work towards achieving sustainability or a need for sustainability can change the governance. Maqueda et al. [ 54 ] investigated an intervention project in the Ecuadorian Andes. They investigated the impact on sustainability before and after an intervention by applying the framework for the evaluation of natural resource management systems (MESMIS). Infrastructure networks, such as those for energy, transportation and telecommunications, perform key functions for society. Systems have largely been developed and managed in isolation,however, infrastructure now functions as a system of systems, exhibiting complex interdependencies that can leave critical functions vulnerable to failure [ 24 ]. Grafius et al. [ 24 ] argue that research efforts and management strategies have so far focused on risks and negative aspects of the complexity of the systems perspectives,however, their case study review identified how interdependencies can also be seen positively, representing possibilities to increase organisational resilience and sustainability. The integrated social innovation and scenario-thinking mechanism was developed by Bonsu et al. [ 4 ] as a bottom-up tool for empowering citizens, including youth and decision-makers, in delivering sustainable development goals, plans, policies and programmes.

Research has become more interested in how humanity can work towards the prosperous development of civilisation. Echaubard et al. [ 17 ] refer to socio-ecological systems theory to describe resilience, as this remains the best operational framework for meeting the need for integration and adaptive governance to obtain sustainable development goals. The Anthropocene reality of rising system-wide turbulence calls for transformative change towards a sustainable future. Emerging technologies and social innovations work towards a more resilient biosphere and are considered as essential parts of such transformations [ 20 ]. Tim et al. [ 88 ] introduced the concept of digital social innovation. They studied e-commerce to bring people out of poverty and how both bottom-up and top-down interventions can be instrumental in overcoming the bottlenecks to developing a resilient community. Gender equality and empowerment are considered as indicators contributing towards the natural sustainability of the system [ 54 ]. In their work on a case study of the watchmaking region of the Swiss Jura, Jeannerat [ 30 ] introduce the concept of ‘valuation’ to interpret the contemporary territorial dynamics of innovation.

Stakeholders and how they are related to technology are the central part of interconnected systems. Kok et al. [ 35 ] refer to this as human and non-human agency. Maqueda et al. [ 54 ] and Echaubard et al. [ 17 ] investigated communities as stakeholders. Jeannerat [ 30 ] looked into whole industries as stakeholders, while Grafius et al. [ 24 ] consider a multitude of infrastructure stakeholders including the energy, ICT, transportation, waste and water sectors, while also including stakeholders from academia and governance. Kok et al. [ 35 ] consider that both macro- and micro-scales need to be studied to consider the properties of wholes as well as those of parts of the system. Researchers consider spatial scales by referring to large [ 35 ] and small scales [ 20 , 67 ]. Territorial scales local vs global scale are used by Jeannerat [ 30 ], with Folke et al. [ 20 ] adding planetary scale. Cross-scale scale [ 4 , 17 ] is also considered by the researchers.

The aim of this paper was to highlight the interrelations among resilience, sustainability and innovation, identify the major and emerging themes in the literature and provide an understanding as to how they are studied in practice and identify future research directions.

Interrelations Among Sustainability, Resilience and Innovation

To answer the first research question on the interrelations of the topics of resilience, sustainability and innovation, our study showed that there is a strong overlap among the constructs, with authors on multiple occasions using the concepts interchangeably or without specifically distinguishing them, only referring to interactions among socio-ecological and economic systems. A majority of the authors identify resilience as a potential pathway to achieving sustainability [ 1 , 10 , 13 , 14 , 63 , 69 , 97 ] or a trait of sustainability [ 54 ], while innovation is presented as the way to achieve both [ 1 , 13 , 14 , 18 , 19 , 21 ,  69 , 71 , 97 , 98 ]. The strong overlap between sustainability and resilience can already be seen in the definitions of the terms. For example, Reidsma et al. [ 76 ] define resilience as the ‘ability to ensure the provision of the system functions in the face of complex and accumulating economic, social, environmental and institutional shocks and stresses’. The definition of sustainability, on the other hand, considers the adequate performance of the system across economic, social and environmental domains [ 76 , 83 ]. Therefore, sustainability can be considered as an umbrella under which resilience can be studied more specifically when referring to interactions between socio-ecological and economic systems. Resilience can mean ensuring adequate performance of the system when faced with shocks and stressors, while sustainability can ensure the adequate performance of the system in general. This could explain why in 2019–2021, the thematic evolution showed there was a move in the research from resilience, which was a major theme of interest from 1999 to 2018, towards sustainability and other topics, such as vulnerability, policy, knowledge, adaptation and impact. Innovation, on the other hand, was not one of the themes identified in the examination of the thematic evolution, which could be explained by the fact that innovation is involved in the processes of obtaining both, and therefore, no specific distinguishment is present.

Resilience encompasses both aspects of adaptation and transformation if it is referred to in the context of sustainability [ 14 ,  37 , 38 , 70 , 97 ] while Reidsma et al. [ 76 ] also add the robustness of the system under the resilience domain. Robustness can also refer to the negative side of resilience. Olsson et al. [ 70 ] call for a clear distinction between resilience and robustness in this regard. Newer research merges the two concepts of transformation and adaptation together into transformational adaptation when referring to resilience [ 33 ]. The literature differs when it comes to social [ 57 , 60 , 62 , 63 , 70 , 97 , 98 ], technological [ 70 , 98 ] and a hybrid version of socio-digital innovation [ 88 ], and sustainability requires adequate performance across economic, social and environmental domains [ 76 , 83 ]. As such, one can argue that resilience and innovation refer to the processes, while sustainability is concerned with the outcomes [ 76 , 83 ]. Research on integrating the three concepts of sustainability, resilience and innovation shows promising results, with the three concepts being used interchangeably or in relation to each other. Resilience can work as a catalyst in maintaining the sustainability of the system when faced with external pressures and shocks, while sustainability can be considered as a broader concept, which requires adequate functioning of the system even when no external shocks or stressors occur. Furthermore, our overview of the literature suggests that two theories are used when referring to resilience, sustainability and innovation by many of authors – the theories of closing [ 9 , 61 , 74 ] and adaptive cycles [ 5 , 13 , 14 , 57 , 97 ]. The theory of the adaptive cycle represents the most used theory when looking into processes related to exploitation, conservation, release and reorganisation. The resilience of the system enables the system to continue in an unproblematic manner through the adaptive cycle [ 14 ,  37 , 38 , 97 ]. The second theory that often appears in the literature, that of the closing cycle, has become more popular in recent years, and especially in relation to the circular economy and minimising waste in the environment [ 9 , 61 , 74 ].

Stakeholders

If we wish to achieve a large-scale change towards utilising innovation to foster resilience and sustainability, then we have to consider the multitude of actors within systems and their interactions in the implementation. To answer the second research question, our review of the literature found that researchers rarely involve one type of stakeholder in their analysis of sustainability, resilience and innovation. The stakeholders range from individual, organisational and institutional stakeholders [ 7 ], while a newly emerging theme on interconnected systems proposes the importance of linking sustainability, resilience and innovation to both human and non-human agency [ 35 ]. With regard to the major themes of research, we have seen how complex and important the stakeholders involved in the process are due to the interdisciplinary nature and dependency of the field. We may classify stakeholders in terms of environmental protection organisations [ 12 , 69 ], farms and agriculture organisations [ 1 , 40 , 69 ], non-profit organisations [ 13 , 18 , 19 , 98 ], educational institutions [ 13 ], business organisations [ 40 ], infrastructure organisations [ 24 ] and governmental organisations [ 7 , 21 ,  40 , 60 , 62 , 70 , 98 ].

Another classification can be based on the individual level [ 6 , 7 ,  63 , 71 ] such as employees [ 66 ] or customers [ 9 ] and collectives or communities ([ 13 , 17 , 40 , 54 , 63 ,  69 ] institutions and organisations [ 6 , 32 , 57 , 70 , 97 ], as well as industries [ 30 ]. In some instances, the researchers consider social, economic and state actors [ 60 , 62 ]. Both top-down and bottom-up approaches can be considered as sources of transformational innovation, which is why different stakeholders at different positions of power should not be neglected while studying resilience, sustainability and innovation [ 23 , 98 ]. The reason why so many different stakeholders were included in the analysis or referred to in the literature could be due to the importance of the inter-connectedness and dependence of different actors within the fields of sustainability, resilience and innovation, which requires us to study the related concepts across multiple stakeholders [ 12 ]. Our research shows that sustainability, resilience and innovation can be applied simultaneously by considering the multitude of stakeholders involved in the process. The results also showed the need to consider multi-stakeholder systems and how they interact with each other.

Methods and Scales

The third research question was concerned with which methods are used by researchers in studying resilience, sustainability and innovation. Literature reviews were the main method used, followed by case studies. Among the major themes, there were only two quantitative and qualitative studies in the works examined, while researchers also used example reviews. In recent years within the emerging themes, there has been a shift towards more sophisticated methods, such as systematic literature review, comparative analysis of investment criteria and surveys, although literature reviews and case studies still dominate the research.

Our review of the literature showed that changes in one system may only be achieved on certain occasions by if there are also changes in another system. Our fourth research question was therefore concerned with what scales the research should be conducted at. Researchers emphasise the importance of studying the concepts across different scales or including a multi-scale approach [ 6 , 7 ,  60 , 62 , 63 , 70 ], as this is fundamental to the interplay between persistence, change, adaptability and transformability. Researchers have used several different scales referring to the spatial (large, medium, small), territorial (organisational, local, regional, national, international, planetary), temporal (short-term, mid-term, long-term) and functional. Researchers have also considered classifications on the micro-, meso- and macro-scales [ 57 , 98 ].

Major and Emerging Themes of Research

Concepts of resilience, sustainability and innovation are often used as solutions to the most pressing issues that humanity is facing; this can also be seen in the topics for major and emerging themes. To answer the fourth research question, our literature review identified three major themes: ‘transformational innovation’, ‘political governance’ and ‘socio-ecological systems’. The first major theme of transformational innovation emerged as a type of innovation that can establish the move to resilient and sustainable systems. The reason why this theme emerged could be due to the rising importance of linking innovation to ecological integrity [ 98 ] and socio-ecological transformation [ 63 , 70 , 71 , 97 ]. This theme also emerged due to relatability to the concepts of resilience to transformation trait as necessary to be included in contributing towards sustainability [ 14 ,  37 , 38 , 70 , 97 ]. The second major theme of political governance studied how policymaking can be guided to implement innovation or to achieve resilience and sustainability. Political governance is crucial, although while it may present the pathway towards achieving resilience and sustainability of the system, it has also shown limited results in Butler et al. [ 6 , 7 ]. This theme could have emerged due to the importance of linking political governance in implementing resilience, sustainability and innovation [ 12 ]. In fact, a study by Feola and Nunes [ 18 , 19 ] clearly showed the importance of initiatives to be supported by the policymakers to be successful. Finally, research on socio-ecological systems showcases the importance of studying these concepts from both the social and ecological dimensions. For example, Cumming et al. [ 13 ] placed the social and ecological dimensions under the definition of resilience. Such a definition is possible when referring to resilience in the context of sustainability, which encompasses both social and ecological dimensions [ 1 , 13 , 14 ].

Our article also identified three emerging themes: ‘food security and agriculture’, ‘business and finance’ and ‘interconnected systems’. The first of these, on food security and agriculture, was the theme with the highest number of research articles. The reason being could be due to the growing need to assess and design solutions necessary to combat food insecurity, as well as to get to reach sustainable development goals such as ending world hunger by 2030 [ 67 ]. There is a shift into resilience-focused eco-friendly agriculture [ 16 ], which could explain why the need for research in this domain rises in its importance. The second emerging theme referred to business and finance. Research in this domain has started to introduce the concept of organisational resilience, as well as the concepts of circular economy ([ 9 , 74 , 61 ]. The need to consider the resilience of the system has shown to be especially crucial during the COVID-19 pandemic, which is why research has looked to digital technology can improve resilience aftershocks in a-post COVID-19 context [ 3 ,  24 , 43 , 90 ]. The third emerging theme of interconnected systems reveals the importance of considering sustainability, resilience and innovation from the systems perspective, and how human agents and non-human agents interact. There is thus a move from looking at individual parts to the system as researchers start to consider how individual parts are nested inside the overall system [ 35 ]. The importance of interdependency begins to be an issue within this emerging theme, when interdependencies of the system can also play a positive role in influencing the system towards achieving resilience [ 24 ].

Limitations and Areas of Improvement

This literature review is limited to the interrelations among the concepts of sustainability, resilience and innovation, and thus, it did not include studies that investigated these separately. Our study only reviewed the most highly cited articles and the articles within the emerging themes and thus did not consider previous research that could have examined these concepts but was not highly cited or recent enough to be included in the review. This is especially true for the research on methods. It may be the case that literature reviews are more highly cited than empirical research, and therefore, our study did not identify more research that uses an empirical approach to study these concepts. Moreover, due to the complex nature of urban planning, infrastructure and other related concepts, the current study did not consider research in these areas that investigated sustainability, resilience and innovation in different domains. There is a need for more empirical work looking into the three concepts of sustainability, resilience and innovation together.

Future Research Directions

In the following sections, we highlight opportunities for knowledge integration and new research directions. Given the substantial overlap among the three concepts, the question arises as to whether sustainability, innovation and resilience could be integrated and in what ways can they be researched together. Future research is needed to study the propositions of inter-relatedness of the topics empirically by applying quantitative studies that would compare the systems response and involvement in sustainability, resilience and innovation across multiple cases and stakeholders. Furthermore, longitudinal research is needed to study these concepts in practice. In the next section, we review potential new directions in research on these concepts.

Future research on transformational innovation could examine how technological innovation has caused a change in certain environmental outcomes (CO 2 emissions, atmospheric temperature and rising sea levels). Research is necessary to identify how transformational innovation can contribute to longitudinal changes in socio-ecological systems, and what can be done when innovation reaches the routinised phase. Much more research is needed on how to classify the differences between harmful and non-harmful innovation [ 57 ]. A coherent theory could also be developed on the emergence of transformational innovation [ 70 , 71 ].

The researchers working on this theme of political governance mainly focused on developing countries [ 6 , 7 ], while research on developed countries remains limited. Moreover, Feola and Nunes [ 18 , 19 ] argue that studies on the dynamic nature of local and global linkages are rare, and their work was the only one among the highly cited articles included in our research that performed such investigation. More research is necessary on support by policymakers [ 18 , 19 ]. Future research could also explore factors contributing towards the emergence of bad resilience and the role power plays in implementing resilience towards achieving sustainability and innovation.

The frameworks developed by Darnhofer et al. [ 14 ] and Cumming et al. [ 13 ] can be used by researchers to study the concept of resilience in practice, although more research is needed to test these approaches. Future research should identify which feedbacks from the interactions between socio-ecological systems can shape the transformation towards achieving resilience and sustainability [ 13 , 14 ]. The results of such work could be used to identify how human factors interact with the environmental aspects [ 14 ].

With climate change, the interest in the topics of food security and agriculture is likely to rise in its importance. More research should be done to explore how innovation can potentially address the challenges posed by climate change in this context of food and agriculture. Even though changes in consumer preferences have started to attract the interest of researchers, research work on the policy agendas that contribute towards more sustainable food consumption choices remains limited [ 81 ]. Future research could explore how farming systems evolve to integrate consumption choices in their decision-making.

Research on business and finance could, in the future, consider both sides of the coin – what it takes to implement sustainability, resilience and innovation in practice, as well as what this means in terms of ecological, economic and social changes as such actions are taken. It is important to define and agree on the measure of sustainability; however, Carraresi and Broring [ 9 ] argue that no such measure has been proposed that would include both the elements of environmental as well as financial sustainability. Research could explore how can the two processes of the adaptative and closing cycles interact and contribute towards sustainable, resilient and innovative outcomes for organisations.

Future research could test the propositions of systems, as proposed by Kok et al. [ 35 ]. The research on interconnected systems considers that specific community-level interventions can offset certain changes on a societal level. More research is needed on how the individual-level factors such as belief in free will, attitudes, autonomous motivation, norms or learning can contribute towards a system that is innovative, resilient and sustainable. Kok et al. [ 35 ] also mention non-human agency, and much more research should be done to define the interactions between humans and technology in building adaptive and transformational systems that are sustainable.

The systematic review of the literature presented in this work shows there is considerable potential regarding studying resilience, innovation and sustainability together, as there is a great overlap among these concepts. Innovation is important for obtaining both resilience and sustainability, while resilience is involved in processes towards achieving and maintaining sustainability. Resilience is more concerned with processes and sustainability with outcomes. We identified three major themes in the literature: ‘socio-ecological systems’, ‘transformational innovation’ and ‘political governance’. We also found three emerging research streams: ‘food security and agriculture’, ‘business and finance’ and ‘interconnected systems’. The results indicate that multi-scale and multi-stakeholder approaches should be adopted when studying resilience, sustainability and innovation simultaneously. The predominant methods for studying these concepts in the literature were case studies and literature reviews. There remains a need for cross-scale and multilevel empirical quantitative studies that would investigate how these concepts are applied and work longitudinally and across different stakeholders and organisational levels. Future research should consider the human aspects of implementation, what fosters and what creates a barrier in implementation.

Data availability

All the abstracts of the articles used in the review of this study have been made available in the Supporting information.

Baker S, Mehmood A (2015) Social innovation and the governance of sustainable places. Local Environ 20(3):321–334. https://doi.org/10.1080/13549839.2013.842964

Article   Google Scholar  

Benitez B et al (2020) Empowering women and building sustainable food systems: a case study of Cuba’s local agricultural innovation project. Front Sustain Food Syst 4(554414):1–17. https://doi.org/10.3389/fsufs.2020.554414

Bhaskar S et al (2020) At the epicenter of COVID-19-the tragic failure of the global supply chain for medical supplies. Front Public Health 8. https://doi.org/10.3389/fpubh.2020.562882

Bonsu NO, TyreeHageman J, Kele J (2020) Beyond agenda 2030: future-oriented mechanisms in localising the sustainable development goals (SDGs). Sustainability 12(23):1–21. https://doi.org/10.3390/su12239797

Boyer J (2020) Toward an evolutionary and sustainability perspective of the innovation ecosystem: revisiting the panarchy model. Sustainability 12(8):1–17. https://doi.org/10.3390/su12083232

Butler JRA et al (2014) Framing the application of adaptation pathways for rural livelihoods and global change in eastern Indonesian islands. Glob Environ Chang 28:368–382. https://doi.org/10.1016/j.gloenvcha.2013.12.004

Butler JRA et al (2016) Priming adaptation pathways through adaptive co-management: design and evaluation for developing countries. Clim Risk Manag 12:1–16. https://doi.org/10.1016/j.crm.2016.01.001

Campbell A (2020) Australian rangelands science – a strategic national asset. Rangel J 42(5):261–264. https://doi.org/10.1071/RJ20062

Carraresi L, & Broring S (2021) How does business model redesign foster resilience in emerging circular value chains? J Clean Prod 289:125823. https://doi.org/10.1016/j.jclepro.2021.125823

Castro-Arce K, Parra C, Vanclay F (2019) Social innovation, sustainability, and the governance of protected areas: revealing theory as it plays out in practice in Costa Rica. J Environ Planning Manage 62(13):2255–2272. https://doi.org/10.1080/09640568.2018.1537976

Chang W et al (2018) Shiny: web application framework for R (R package version 1.2.0). Retrieved on 1st of July 2021: https://CRAN.R-project.org/package=shiny

Chapin FS et al (2006) Policy strategies to address sustainability of Alaskan boreal forests in response to a directionally changing climate. Proc Natl Acad Sci USA 103(45):16637–16643. https://doi.org/10.1073/pnas.0606955103

Article   CAS   Google Scholar  

Cumming GS et al (2005) An exploratory framework for the empirical measurement of resilience. Ecosystems 8(8):975–987. https://doi.org/10.1007/s10021-005-0129-z

Darnhofer I, Fairweather J, Moller H (2010) Assessing a farm’s sustainability: insights from resilience thinking. Int J Agric Sustain 8(3):186–198. https://doi.org/10.3763/ijas.2010.0480

Del Giudice M, Di Vaio A, Hassan R, Palladino R (2021) Digitalization and new technologies for sustainable business models at the ship-port interface: a bibliometric analysis. Marit Policy Manag. https://doi.org/10.1080/03088839.2021.1903600

Dong LX (2021) Toward resilient agriculture value chains: challenges and opportunities. Prod Oper Manag 30(3):666–675. https://doi.org/10.1111/poms.13308

Echaubard P et al (2020) Fostering social innovation and building adaptive capacity for dengue control in Cambodia: a case study. Infect Dis Poverty 9(1):1–12. https://doi.org/10.1186/s40249-020-00734-y

Feola G, Nunes R (2014a) Jan). Success and failure of grassroots innovations for addressing climate change: the case of the Transition Movement. Glob Environ Chang 24:232–250. https://doi.org/10.1016/j.gloenvcha.2013.11.011

Feola G, Nunes JR (2014b) Success and failure of Grassroots Innovations for addressing climate change: the case of the Transition Movement. Glob Environ Chang 24:232–250

Folke C, Polasky S, Rockstrom J, Galaz V, Westley F, Lamont M, … Walker BH (2021) Our future in the Anthropocene biosphere. Ambio 50(4):834-869. https://doi.org/10.1007/s13280-021-01544-8

Galaz V (2012) Geo-engineering, governance, and social-ecological systems: critical issues and joint research needs. Ecol Soc 17(1):24. https://doi.org/10.5751/ES-04677-170124

Galaz V, Olsson P, Hahn T, Folke C, Svedin U (2008) The problem of fit among biophysical systems, environmental and resource regimes, and broader governance systems: insights and emerging challenges. In: Young OR, King LA, Schroeder H (eds) Institutions and environmental change: principal findings, applications, and research frontiers. MIT Press, Cambridge, pp 147–186

Chapter   Google Scholar  

Ghazinoory S, Sarkissian A, Farhanchi M, Saghafi F (2020) Renewing a dysfunctional innovation ecosystem: the case of the Lalejin ceramics and pottery. Technovation 96–97. https://doi.org/10.1016/j.technovation.2020.102122

Grafius DR, Varga L, Jude S (2020) Infrastructure interdependencies: opportunities from complexity. J Infrastruct Syst 26(4). https://doi.org/10.1061/(ASCE)IS.1943-555X.0000575

Habiyaremye A (2021) Co-operative learning and resilience to COVID-19 in a small-sized South African enterprise. Sustainability 13(4):1–17. https://doi.org/10.3390/su13041976

Holling CS (1986) The resilience of terrestrial ecosystems; local surprise and global change. In: Clark WC, Munn RE (eds) Sustainable development of the biosphere, vol 10. Cambridge University Press, Cambridge (UK), pp 292–317

Google Scholar  

Ilieva RT, Hernandez A (2018) Scaling-up sustainable development initiatives: a comparative case study of agri-food system innovations in Brazil, New York, and Senegal. Sustainability 10(11):1–17. https://doi.org/10.3390/su10114057

Janssen MA (2007) An update on the scholarly networks on resilience, vulnerability, and adaptation within the human dimensions of global environmental change. Ecol Soc 12(2):9. http://www.ecologyandsociety.org/vol12/iss2/art9/

Janssen MA, Schoon ML, Ke W, Börner K (2006) Scholarly networks on resilience, vulnerability and adaptation within the human dimensions of global environmental change. Glob Environ Chang 16:240–252

Jeannerat H (2021) From territorial dynamics of innovation to territorial dynamics of evaluation. Revue D Economie Regionale Et Urbaine 1:29–50

Justice V, Bhaskar P, Pateman H, Cain P, Cahoon S (2016) US container port resilience in a complex and dynamic world. Marit Policy Manag 43(2):179–191. https://doi.org/10.1080/03088839.2015.1133937

Karman A, Savaneviciene A (2021) Enhancing dynamic capabilities to improve sustainable competitiveness: insights from research on organisations of the Baltic region. Balt J Manag 16(2):318–341. https://doi.org/10.1108/BJM-08-2020-0287

Kasdan M, Kuhl L, & Kurukulasuriya P (2019) The evolution of transformational change in multilateral funds dedicated to financing adaptation to climate change. Climate Dev 13(5):427–442. https://doi.org/10.1080/17565529.2020.1790333

Ko YK, Barrett BFD, Copping AE, Sharifi A, Yarime M, Wang X (2019) Energy transitions towards low carbon resilience: evaluation of disaster-triggered local and regional cases. Sustainability 11(23):1–23. https://doi.org/10.3390/su11236801

Kok KPW, Loeber AMC, Grin J (2021) Politics of complexity: conceptualizing agency, power and powering in the transitional dynamics of complex adaptive systems. Res Policy 50(3):1–11. https://doi.org/10.1016/j.respol.2020.104183

Landholm DM, Holsten A, Martellozzo F, Reusser DE, Kropp JP (2019) Climate change mitigation potential of community-based initiatives in Europe. Reg Environ Change 19(4):927–938. https://doi.org/10.1007/s10113-018-1428-1

Larsson M, Milestad R, Hahn T, von Oelreich J (2016a) The resilience of a sustainability entrepreneur in the Swedish food system. Sustainability 8(550):1–18. https://doi.org/10.3390/su8060550

Larsson M, Milestad R, Hahn T, von Oelreich J (2016b) The resilience of a sustainability entrepreneur in the Swedish food system. Sustainability 8(6):1–18. https://doi.org/10.3390/su8060550

Le Bellec F, Rajaud A, Ozier-Lafontaine H, Bockstaller C, Malezieux E (2012) Evidence for farmers’ active involvement in co-designing citrus cropping systems using an improved participatory method. Agron Sustain Dev 32(3):703–714. https://doi.org/10.1007/s13593-011-0070-9

Leach M et al (2012) Transforming innovation for sustainability. Ecol Soc 17(2):11. https://doi.org/10.5751/ES-04933-170211

Lebel L et al (2006) Governance and the capacity to manage resilience in regional social-ecological systems. Ecol Soc 11(1):19. http://www.ecologyandsociety.org/vol11/iss1/art19/

Leopold M, Thebaud O, Charles A (2019) The dynamics of institutional innovation: crafting co-management in small-scale fisheries through action research. J Environ Manage 237:187–199. https://doi.org/10.1016/j.jenvman.2019.01.112

Lepore D, Micozzi A, Spigarelli F (2021) Industry 4.0 accelerating sustainable manufacturing in the COVID-19 era: assessing the readiness and responsiveness of Italian regions. Sustainability 13(5):1–19. https://doi.org/10.3390/su13052670

Lewison RL et al (2016) How the DPSIR framework can be used for structuring problems and facilitating empirical research in coastal systems. Environ Sci Policy 56:110–119. https://doi.org/10.1016/j.envsci.2015.11.001

Linnenluecke MK (2015) Resilience in business and management research: a review of influential publications and a research agenda. Int J Manag Rev. https://doi.org/10.1111/ijmr.12076

Linnenluecke M, Smith T & McKnight B (2016) Environmental finance: a research agenda for interdisciplinary finance research. Economic Modelling 59:124–130.  https://doi.org/10.1016/j.econmod.2016.07.010

Linnenluecke MK (2017) Resilience in business and management research: a review of influential publications and a research agenda. Int J Manag Rev 19(1):4–30. https://doi.org/10.1111/ijmr.12076

Linnenluecke MK, Chen X, Ling X, Smith T, Zhu Y (2017) Research in finance: a review of influential publications and a research agenda. Pac Basin Financ J 43:188–199. https://doi.org/10.1016/j.pacfin.2017.04.005

Linnenluecke M, Marrone M, Singh AK (2020) Conducting systematic literature reviews and bibliometric analyses. Aust J Manag 45(2):175–194. https://doi.org/10.1177/0312896219877678

Liu YY, Ji D, Zhang L, An JJ, Sun WY (2021) Rural financial development impacts on agricultural technology innovation: evidence from China. Int J Environ Res Public Health 18(3):1–17. https://doi.org/10.3390/ijerph18031110

Lockshin L, & Corsi AM (2020) Key research topics likely to generate Australian and other wine producer countries’ support during the period 2020–2030. Int J Wine Bus Res 32(4):493–502. https://doi.org/10.1108/IJWBR-01-2020-0004

Lubell M, Hillis V, Hoffman M (2011) Innovation, cooperation, and the perceived benefits and costs of sustainable agriculture practices. Ecol Soc 16(4). https://doi.org/10.5751/ES-04389-160423

Lund AJ, Lopez-Carr D, Sokolow SH, Rohr JR, De Leo GA (2021) Agricultural innovations to reduce the health impacts of dams. Sustainability 13(4):23. https://doi.org/10.3390/su13041869

Maqueda RH et al (2021) Assessment of the impact of an international multidisciplinary intervention project on sustainability at the local level: case study in a community in the Ecuadorian Andes. Environ Dev Sustain 23:8836–8856. https://doi.org/10.1007/s10668-020-00997-3

Maqueira YM, Sal AG, Valdes EMF, Barrio AMC, Diaz-Maroto IJ (2020) Farmer communities’ adaptative response in front of climatic changes’ effects, Vinales National Park. Avances 22(3):373–387. Retrieved on 1st of May 2021 from https://www.environment.gov.au/system/files/resources/fa9235b0-e42f-4da7-93a9-61e0e3e80d38/files/boodereestrategy.pdf

Mastrolonardo L, Radogna D, Romano M (2018) Resilience and transformation strategies for a becoming housing quality. Techne-J Technol Archit Environ 15:311–322. https://doi.org/10.13128/Techne-22122

McCarthy DDP, Whitelaw GS, Westley FR, Crandall DD, Burnett D (2014) The Oak Ridges Moraine as a social innovation: strategic vision as a social-ecological interaction. Ecol Soc 19(1):48. https://doi.org/10.5751/ES-06212-190148

Mills E (2009) A global review of insurance industry responses to climate change. Geneva Pap Risk Insur-Issues Pract 34(3):323–359. https://doi.org/10.1057/gpp.2009.14

Missimer M, Robert KH, Broman G (2017) A strategic approach to social sustainability - part 1: exploring the social system. J Clean Prod 140(1):32–41. https://doi.org/10.1016/j.jclepro.2016.03.170

Moore AW, King L, Dale A, Newell R (2018a) Toward an integrative framework for local development path analysis. Ecol Soc 23(2):1210–1224. https://doi.org/10.5751/ES-10029-230213

Moore EA, Russell JD, Babbitt CW, Tomaszewski B, Clark SS (2020) Spatial modeling of a second-use strategy for electric vehicle batteries to improve disaster resilience and circular economy. Resour Conserv Recycl 160:104889. https://doi.org/10.1016/j.resconrec.2020.104889

Moore ML, Olsson P, Nilsson W, Rose L, Westley FR (2018b) Navigating emergence and system reflexivity as key transformative capacities: experiences from a Global Fellowship program. Ecol Soc 23(2):38. https://doi.org/10.5751/ES-10166-230238

Moore ML et al (2014) Studying the complexity of change: toward an analytical framework for understanding deliberate social-ecological transformations. Ecol Soc 19(4):54. https://doi.org/10.5751/ES-06966-190454

Nicoll K, Zerboni A (2020) Is the past key to the present? Observations of cultural continuity and resilience reconstructed from geoarchaeological records. Quatern Int 545:119–127. https://doi.org/10.1016/j.quaint.2019.02.012

Nicolosi E, French J, Medina R (2020) Add to the map! Evaluating digitally mediated participatory mapping for grassroots sustainabilities. Geogr J 186(2):142–155. https://doi.org/10.1111/geoj.12315

Nyaupane GP, Prayag G, Godwyll J, White D (2020) Toward a resilient organization: analysis of employee skills and organization adaptive traits. J Sustain Tour 29(4):658–677. https://doi.org/10.1080/09669582.2020.1822368

Okello M, Lamo J, Ochwo-Ssemakula M, Onyilo F (2020) Challenges and innovations in achieving zero hunger and environmental sustainability through the lens of sub-Saharan Africa. Outlook Agric 50(2):141–147. https://doi.org/10.1177/0030727020975778

Oktari RS, Shiwaku K, Munadi K, Syamsidik S, Shaw R (2015) A conceptual model of a school-community collaborative network in enhancing coastal community resilience in Banda Aceh, Indonesia. Int J Disaster Risk Reduct 12:300–310. https://doi.org/10.1016/j.ijdrr.2015.02.006

Olsson P, Folke C, Hughes TP (2008) Navigating the transition to ecosystem-based management of the Great Barrier Reef, Australia. Proc Natl Acad Sci USA 105(28):9489–9494. https://doi.org/10.1073/pnas.0706905105

Olsson P, Galaz V, Boonstra WJ (2014) Sustainability transformations: a resilience perspective. Ecol Soc 19(4):1. https://doi.org/10.5751/ES-06799-190401

Olsson P, Moore ML, Westley FR, McCarthy DDP (2017) The concept of the Anthropocene as a game-changer: a new context for social innovation and transformations to sustainability. Ecol Soc 22(2):31. https://doi.org/10.5751/ES-09310-220231

Pörtner HO, Roberts D, Tignor MMB, Poloczanska E, Mintenbeck K,... Rama B (2022) Climate Change 2022 Impacts, Adaptation and Vulnerability. IPCC: Switzerland. Retrieved from  https://www.ipcc.ch/report/ar6/wg2/downloads/report/IPCC_AR6_WGII_SummaryForPolicymakers.pdf

Rahman HMT, Robinson BE, Ford JD, Hickey GM (2018) How do capital asset interactions affect livelihood sensitivity to climatic stresses? Insights from the Northeastern Floodplains of Bangladesh. Ecol Econ 150:165–176. https://doi.org/10.1016/j.ecolecon.2018.04.006

Ramirez J et al (2021) Wastes to profit: a circular economy approach to value-addition in livestock industries. Anim Prod Sci 61(6):541–550. https://doi.org/10.1071/AN20400

Redman CL (2014) Should sustainability and resilience be combined or remain distinct pursuits? Ecol Soc 19(2):37. https://doi.org/10.5751/ES-06390-190237

Reidsma PW et al (2020) How do stakeholders perceive the sustainability and resilience of EU farming systems? EuroChoices 19(2):18–27. https://doi.org/10.1111/1746-692X.12280

Rockström J et al (2009) A safe operating space for humanity. Nature 461:472–475

Ruben R, Verhagen J, Plaisier C (2019) The challenge of food systems research: what difference does it make? Sustainability 11(1):1–14. https://doi.org/10.3390/su11010171

Sanz-Hernandez A (2021) Privately owned forests and woodlands in Spain: changing resilience strategies towards a forest-based bioeconomy. Land Use Policy 100:1–13. https://doi.org/10.1016/j.landusepol.2020.104922

Sanz-Hernandez A, Sanagustin-Fons MV, Lopez-Rodriguez ME (2019) A transition to an innovative and inclusive bioeconomy in Aragon, Spain. Environ Innov Soc Trans 33:301–316. https://doi.org/10.1016/j.eist.2019.08.003

Saviolidis NM et al (2020) Stakeholder perceptions of policy tools in support of sustainable food consumption in Europe: policy implications. Sustainability 12(17):1–23. https://doi.org/10.3390/su12177161

Scheffran J (2016) From a climate of complexity to sustainable peace: viability transformations and adaptive governance in the Anthropocene. In: HG Brauch, U Oswald Spring, J Grin, Scheffran J (eds) Handbook on Sustainability Transition and Sustainable Peace (vol 10). Springer International Publishing Ag, pp 305–346

Seghieri J et al (2020) Research and development challenges in scaling innovation: a case study of the LEAP-Agri RAMSES II project. Agrofor Syst. https://doi.org/10.1007/s10457-020-00532-3

Tan S, De D, Song WZ, Yang JJ, Das SK (2017) Survey of security advances in smart grid: a data driven approach. IEEE Commun Surv Tutorials 19(1):397–422. https://doi.org/10.1109/COMST.2016.2616442

Tawfik GM, Surya Dila KA, Fadlelmola Mohamed MY, Hien Tam DN, Dang Kien N, Ahmed AM, Huy NT (2019) A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health 47(46):1–9. https://doi.org/10.1186/s41182-019-0165-6

Teles MD, de Sousa JF (2018) Linking fields with GMA: sustainability, companies, people, and operational research. Technol Forecast Soc Chang 126:138–146. https://doi.org/10.1016/j.techfore.2017.05.012

Tengberg A, Valencia S (2018) Integrated approaches to natural resources management–theory and practice. Land Degrad Dev 29(6):1845–1857. https://doi.org/10.1002/ldr.2946

Tim YN, Cui LL, Sheng ZZ (2021) Digital resilience: how rural communities leapfrogged into sustainable development. Inf Syst J 31(2):323–345. https://doi.org/10.1111/isj.12312

Tolosana ES, Serrano JO (2020) Social strategies, crisis and rural governance. Paradoxes of resilience in the Pyrenees Mountains, Navarra. Cuadernos De Desarrollo Rural 17(85). https://doi.org/10.11144/Javeriana.cdr17.escg

Trivellas P, Malindretos G, Reklitis P (2020) Implications of green logistics management on sustainable business and supply chain performance: evidence from a survey in the Greek agri-food sector. Sustainability 12(24):1–29. https://doi.org/10.3390/su122410515

Turetta APD, Bonatti M, Sieber S (2021) Resilience of community food systems (CFS): co-design as a long-term viable pathway to face crises in neglected territories? Foods 10(3):1–8. https://doi.org/10.3390/foods10030521

United Nations Brundtland Commission (1987). Report of the World Commission on Environment and Development: our common future . Oslo, Norway: United Nations General Assembly, Development and International Co-operation: Environment. Retrieved on 4 th of July 2021 from http://www.un-documents.net/our-common-future.pdf

Vlasov M, Bonnedahl KJ, Vincze Z (2018) Entrepreneurship for resilience: embeddedness in place and in trans-local grassroots networks. J Enterp Commun-People Places Glob Econ 12(3):374–394. https://doi.org/10.1108/JEC-12-2017-0100

Wang H, Wang Q, Sheng X (2021) Does corporate financialization have a non-linear impact on sustainable total factor productivity? Perspectives of cash holdings and technical innovation. Sustainability 13(5):1–17. https://doi.org/10.3390/su13052533

Warman R (2019) Resilience, innovation and sustainability in resource use. In: Warman R (ed) Forest Ecosystem Management and Timber Production: Divergence and Resource Use Resilience. Routledge, pp 173–187

Weaver DB, Moyle B, McLennan CLJ (2021) The citizen within: positioning local residents for sustainable tourism. J Sustain Tour. https://doi.org/10.1080/09669582.2021.1903017

Westley FR et al (2013) A theory of transformative agency in linked social-ecological systems. Ecol Soc 18(3):27. https://doi.org/10.5751/ES-05072-180327

Westley F et al (2011) Tipping toward sustainability: emerging pathways of transformation. Ambio 40(7):762–780. https://doi.org/10.1007/s13280-011-0186-9

Winder N, McIntosh BS, Jeffrey P (2005) The origin, diagnostic attributes, and practical application of co-evolutionary theory. Ecol Econ 54(4):347–361. https://doi.org/10.1016/j.ecolecon.2005.03.017

World Meterological Organization (2021) State of the Global Climate 2020 (Report No. 1264). Retrieved 22nd of February 2022 from https://library.wmo.int/doc_num.php?explnum_id=10618

Worstell J, Green J (2017) Eight qualities of resilient food systems: toward a sustainability/resilience index. J Agric Food Syst Commun Dev 7(3):23–41. https://doi.org/10.5304/jafscd.2017.073.001

Xiang Z, Fesenmaier DR, Werthner H (2020) Knowledge creation in information technology and tourism: a critical reflection and an outlook for the future. J Travel Res 60(6):1371–1376. https://doi.org/10.1177/0047287520933669

Yan WY et al (2020) Rethinking Chinese supply resilience of critical metals in lithium-ion batteries. J Clean Prod 256:120719. https://doi.org/10.1016/j.jclepro.2020.120719

Yasir MA, Amir AM, Maelah R, Nasir AHM (2020) Establishing customer knowledge through customer accounting in tourism industry: a study of hotel sector in Malaysia. Asian J Account Govern 14:155–165. https://doi.org/10.17576/AJAG-2020-14-12

Zhang H, Dolan C, Jing SM, Uyimleshi J, Dodd P (2019) Bounce forward: economic recovery in post-disaster Fukushima. Sustainability 11(23):1–24. https://doi.org/10.3390/su11236736

Zupancic N (2021) Systematic literature review: implementation of resilience, sustainability, and innovation. Retrieved on 1st of May 2022 from osf.io/h95xy

Download references

Acknowledgements

I would like to acknowledge the support from Prof. Dr. Tom Smith at Macquarie University in the preparation of this manuscript.

Author information

Authors and affiliations.

Department of Psychology, University of Ljubljana, Ljubljana, Slovenia

N. Zupancic

You can also search for this author in PubMed   Google Scholar

Contributions

This article is the work of the author named on the submission of the manuscript. The author of this manuscript has drafted the work or revised it critically, approved the version to be published and agreed to be accountable for all aspects of the work, including that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Corresponding author

Correspondence to N. Zupancic .

Ethics declarations

Competing interests.

The author declares no competing interests.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (TXT 2200 KB)

Rights and permissions.

Reprints and permissions

About this article

Zupancic, N. Systematic Literature Review: Inter-Reletedness of Innovation, Resilience and Sustainability - Major, Emerging Themes and Future Research Directions. Circ.Econ.Sust. 3 , 1157–1185 (2023). https://doi.org/10.1007/s43615-022-00187-5

Download citation

Received : 16 November 2021

Accepted : 15 June 2022

Published : 25 July 2022

Issue Date : September 2023

DOI : https://doi.org/10.1007/s43615-022-00187-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Sustainability
• Find a journal
• Publish with us
• Track your research