A free, AI-powered research tool for scientific literature

  • Oren Etzioni
  • Moral Reasoning
  • Classical Conditioning

New & Improved API for Developers

Introducing semantic reader in beta.

Stay Connected With Semantic Scholar Sign Up What Is Semantic Scholar? Semantic Scholar is a free, AI-powered research tool for scientific literature, based at the Allen Institute for AI.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

OWL logo

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

  • An introductory paragraph that explains what your working topic and thesis is
  • A forecast of key topics or texts that will appear in the review
  • Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
  • Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically Evaluate: Mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

  • Summarize the key findings you have taken from the literature and emphasize their significance
  • Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

  • Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
  • Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
  • Qualitative versus quantitative research
  • Empirical versus theoretical scholarship
  • Divide the research by sociological, historical, or cultural sources
  • Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

  • It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
  • Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
  • Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
  • Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

  • USC Libraries
  • Research Guides

Organizing Your Social Sciences Research Paper

  • 5. The Literature Review
  • Purpose of Guide
  • Design Flaws to Avoid
  • Independent and Dependent Variables
  • Glossary of Research Terms
  • Reading Research Effectively
  • Narrowing a Topic Idea
  • Broadening a Topic Idea
  • Extending the Timeliness of a Topic Idea
  • Academic Writing Style
  • Choosing a Title
  • Making an Outline
  • Paragraph Development
  • Research Process Video Series
  • Executive Summary
  • The C.A.R.S. Model
  • Background Information
  • The Research Problem/Question
  • Theoretical Framework
  • Citation Tracking
  • Content Alert Services
  • Evaluating Sources
  • Primary Sources
  • Secondary Sources
  • Tiertiary Sources
  • Scholarly vs. Popular Publications
  • Qualitative Methods
  • Quantitative Methods
  • Insiderness
  • Using Non-Textual Elements
  • Limitations of the Study
  • Common Grammar Mistakes
  • Writing Concisely
  • Avoiding Plagiarism
  • Footnotes or Endnotes?
  • Further Readings
  • Generative AI and Writing
  • USC Libraries Tutorials and Other Guides
  • Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

  • Give a new interpretation of old material or combine new with old interpretations,
  • Trace the intellectual progression of the field, including major debates,
  • Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
  • Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

  • Place each work in the context of its contribution to understanding the research problem being studied.
  • Describe the relationship of each work to the others under consideration.
  • Identify new ways to interpret prior research.
  • Reveal any gaps that exist in the literature.
  • Resolve conflicts amongst seemingly contradictory previous studies.
  • Identify areas of prior scholarship to prevent duplication of effort.
  • Point the way in fulfilling a need for additional research.
  • Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

  • An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
  • Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
  • An explanation of how each work is similar to and how it varies from the others,
  • Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

  • Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
  • Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
  • Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
  • Persuasiveness -- which of the author's theses are most convincing or least convincing?
  • Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

  • Current Situation : Information necessary to understand the current topic or focus of the literature review.
  • Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
  • History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
  • Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
  • Standards : Description of the way in which you present your information.
  • Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

  • Sources in your literature review do not clearly relate to the research problem;
  • You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
  • Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
  • Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
  • Does not describe the search procedures that were used in identifying the literature to review;
  • Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
  • Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

  • How are they organizing their ideas?
  • What methods have they used to study the problem?
  • What theories have been used to explain, predict, or understand their research problem?
  • What sources have they cited to support their conclusions?
  • How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

  • Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
  • Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
  • Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

  • << Previous: Theoretical Framework
  • Next: Citation Tracking >>
  • Last Updated: Feb 8, 2024 1:57 PM
  • URL: https://libguides.usc.edu/writingguide

Harvey Cushing/John Hay Whitney Medical Library

  • Collections
  • Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

  • Biomedical Databases
  • Global (Public Health) Databases
  • Soc. Sci., History, and Law Databases
  • Grey Literature
  • Trials Registers
  • Data and Statistics
  • Public Policy
  • Google Tips
  • Recommended Books
  • Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

  • Explains the background of research on a topic.
  • Demonstrates why a topic is significant to a subject area.
  • Discovers relationships between research studies/ideas.
  • Identifies major themes, concepts, and researchers on a topic.
  • Identifies critical gaps and points of disagreement.
  • Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

Cover Art

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

  • Make sure your research question is not too broad or too narrow.  Is it manageable?
  • Begin writing down terms that are related to your question. These will be useful for searches later.
  • If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

  • This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

  • use the tabs on this guide
  • Find other databases in the Nursing Information Resources web page
  • More on the Medical Library web page
  • ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

  • Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
  • Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
  • Review the abstracts of research studies carefully. This will save you time.
  • Use the bibliographies and references of research studies you find to locate others.
  • Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
  • Ask your librarian for help at any time.
  • Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

  • What was the research question of the study you are reviewing? What were the authors trying to discover?
  • Was the research funded by a source that could influence the findings?
  • What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
  • Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
  • If there are conflicting studies, why do you think that is?
  • How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

  • Review the abstracts carefully.  
  • Keep careful notes so that you may track your thought processes during the research process.
  • Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
  • << Previous: Recommended Books
  • Last Updated: Jan 4, 2024 10:52 AM
  • URL: https://guides.library.yale.edu/YSNDoctoral

Research Methods

  • Getting Started
  • Literature Review Research
  • Research Design
  • Research Design By Discipline
  • SAGE Research Methods
  • Teaching with SAGE Research Methods

Literature Review

  • What is a Literature Review?
  • What is NOT a Literature Review?
  • Purposes of a Literature Review
  • Types of Literature Reviews
  • Literature Reviews vs. Systematic Reviews
  • Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

  • Summarizes and analyzes previous research relevant to a topic
  • Includes scholarly books and articles published in academic journals
  • Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

  • Help gather ideas or information
  • Keep up to date in current trends and findings
  • Help develop new questions

A literature review is important because it:

  • Explains the background of research on a topic.
  • Demonstrates why a topic is significant to a subject area.
  • Helps focus your own research questions or problems
  • Discovers relationships between research studies/ideas.
  • Suggests unexplored ideas or populations
  • Identifies major themes, concepts, and researchers on a topic.
  • Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
  • Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
  • Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

  • provides thorough knowledge of previous studies; introduces seminal works.
  • helps focus one’s own research topic.
  • identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
  • suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
  • identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
  • helps the researcher avoid repetition of earlier research.
  • suggests unexplored populations.
  • determines whether past studies agree or disagree; identifies controversy in the literature.
  • tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

research literature literature

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

research literature literature

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

  • clearly stated objectives with pre-defined eligibility criteria for studies
  • explicit, reproducible methodology
  • a systematic search that attempts to identify all studies
  • assessment of the validity of the findings of the included studies (e.g. risk of bias)
  • systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

  • << Previous: Getting Started
  • Next: Research Design >>
  • Last Updated: Aug 21, 2023 4:07 PM
  • URL: https://guides.lib.udel.edu/researchmethods

Libraries | Research Guides

Literature reviews, what is a literature review, learning more about how to do a literature review.

  • Planning the Review
  • The Research Question
  • Choosing Where to Search
  • Organizing the Review
  • Writing the Review

A literature review is a review and synthesis of existing research on a topic or research question. A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the literature you have read. 

  • Sage Research Methods Core Collection This link opens in a new window SAGE Research Methods supports research at all levels by providing material to guide users through every step of the research process. SAGE Research Methods is the ultimate methods library with more than 1000 books, reference works, journal articles, and instructional videos by world-leading academics from across the social sciences, including the largest collection of qualitative methods books available online from any scholarly publisher. – Publisher

Cover Art

  • Next: Planning the Review >>
  • Last Updated: Jan 17, 2024 10:05 AM
  • URL: https://libguides.northwestern.edu/literaturereviews

Book cover

Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy pp 1–15 Cite as

Methodological Approaches to Literature Review

  • Dennis Thomas 2 ,
  • Elida Zairina 3 &
  • Johnson George 4  
  • Living reference work entry
  • First Online: 09 May 2023

307 Accesses

The literature review can serve various functions in the contexts of education and research. It aids in identifying knowledge gaps, informing research methodology, and developing a theoretical framework during the planning stages of a research study or project, as well as reporting of review findings in the context of the existing literature. This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis. Review authors should consider the scope of the literature review when selecting a type and method. Being focused is essential for a successful review; however, this must be balanced against the relevance of the review to a broad audience.

  • Literature review
  • Systematic review
  • Meta-analysis
  • Scoping review
  • Research methodology

This is a preview of subscription content, log in via an institution .

Akobeng AK. Principles of evidence based medicine. Arch Dis Child. 2005;90(8):837–40.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alharbi A, Stevenson M. Refining Boolean queries to identify relevant studies for systematic review updates. J Am Med Inform Assoc. 2020;27(11):1658–66.

Article   PubMed   PubMed Central   Google Scholar  

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Article   Google Scholar  

Aromataris E MZE. JBI manual for evidence synthesis. 2020.

Google Scholar  

Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs. 2014;114(3):53–8.

Article   PubMed   Google Scholar  

Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs. 2014;114(5):49–56.

Babineau J. Product review: covidence (systematic review software). J Canad Health Libr Assoc Canada. 2014;35(2):68–71.

Baker JD. The purpose, process, and methods of writing a literature review. AORN J. 2016;103(3):265–9.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):1–12.

Brown D. A review of the PubMed PICO tool: using evidence-based practice in health education. Health Promot Pract. 2020;21(4):496–8.

Cargo M, Harris J, Pantoja T, et al. Cochrane qualitative and implementation methods group guidance series – paper 4: methods for assessing evidence on intervention implementation. J Clin Epidemiol. 2018;97:59–69.

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997;127(5):380–7.

Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Cummings SR, Browner WS, Hulley SB, editors. Designing Clinical Research: An Epidemiological Approach. 4th ed. Philadelphia (PA): P Lippincott Williams & Wilkins; 2007. p. 14–22.

Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. JMLA. 2018;106(4):420.

Ferrari R. Writing narrative style literature reviews. Medical Writing. 2015;24(4):230–5.

Flemming K, Booth A, Hannes K, Cargo M, Noyes J. Cochrane qualitative and implementation methods group guidance series – paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J Clin Epidemiol. 2018;97:79–85.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. 2006;5(3):101–17.

Gregory AT, Denniss AR. An introduction to writing narrative and systematic reviews; tasks, tips and traps for aspiring authors. Heart Lung Circ. 2018;27(7):893–8.

Harden A, Thomas J, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018;97:70–8.

Harris JL, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. J Clin Epidemiol. 2018;97:39–48.

Higgins J, Thomas J. In: Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3, updated February 2022). Available from www.training.cochrane.org/handbook.: Cochrane; 2022.

International prospective register of systematic reviews (PROSPERO). Available from https://www.crd.york.ac.uk/prospero/ .

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Landhuis E. Scientific literature: information overload. Nature. 2016;535(7612):457–8.

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03 .

Chapter   Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.

Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred reporting items for (SR) and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;6:264–9.

Mulrow CD. Systematic reviews: rationale for systematic reviews. BMJ. 1994;309(6954):597–9.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):1–13.

Murphy CM. Writing an effective review article. J Med Toxicol. 2012;8(2):89–90.

NHMRC. Guidelines for guidelines: assessing risk of bias. Available at https://nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias . Last published 29 August 2019. Accessed 29 Aug 2022.

Noyes J, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 1: introduction. J Clin Epidemiol. 2018b;97:35–8.

Noyes J, Booth A, Flemming K, et al. Cochrane qualitative and implementation methods group guidance series – paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018a;97:49–58.

Noyes J, Booth A, Moore G, Flemming K, Tunçalp Ö, Shakibazadeh E. Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods. BMJ Glob Health. 2019;4(Suppl 1):e000893.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Healthcare. 2015;13(3):141–6.

Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses. Res Synth Methods. 2019;10(3):330–42.

Article   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):1–7.

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Brit Med J. 2017;358

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. 2016;355

Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–12.

Tawfik GM, Dila KAS, Mohamed MYF, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47(1):1–9.

The Critical Appraisal Program. Critical appraisal skills program. Available at https://casp-uk.net/ . 2022. Accessed 29 Aug 2022.

The University of Melbourne. Writing a literature review in Research Techniques 2022. Available at https://students.unimelb.edu.au/academic-skills/explore-our-resources/research-techniques/reviewing-the-literature . Accessed 29 Aug 2022.

The Writing Center University of Winconsin-Madison. Learn how to write a literature review in The Writer’s Handbook – Academic Professional Writing. 2022. Available at https://writing.wisc.edu/handbook/assignments/reviewofliterature/ . Accessed 29 Aug 2022.

Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18(20):2693–708.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Yoneoka D, Henmi M. Clinical heterogeneity in random-effect meta-analysis: between-study boundary estimate problem. Stat Med. 2019;38(21):4131–45.

Yuan Y, Hunt RH. Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol. 2009;104(5):1086–92.

Download references

Author information

Authors and affiliations.

Centre of Excellence in Treatable Traits, College of Health, Medicine and Wellbeing, University of Newcastle, Hunter Medical Research Institute Asthma and Breathing Programme, Newcastle, NSW, Australia

Dennis Thomas

Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, Indonesia

Elida Zairina

Centre for Medicine Use and Safety, Monash Institute of Pharmaceutical Sciences, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Parkville, VIC, Australia

Johnson George

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Johnson George .

Section Editor information

College of Pharmacy, Qatar University, Doha, Qatar

Derek Charles Stewart

Department of Pharmacy, University of Huddersfield, Huddersfield, United Kingdom

Zaheer-Ud-Din Babar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Cite this entry.

Thomas, D., Zairina, E., George, J. (2023). Methodological Approaches to Literature Review. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_57-1

Download citation

DOI : https://doi.org/10.1007/978-3-030-50247-8_57-1

Received : 22 February 2023

Accepted : 22 February 2023

Published : 09 May 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-50247-8

Online ISBN : 978-3-030-50247-8

eBook Packages : Springer Reference Biomedicine and Life Sciences Reference Module Biomedical and Life Sciences

  • Publish with us

Policies and ethics

  • Find a journal
  • Track your research
  • Methodology
  • Open access
  • Published: 11 October 2016

Reviewing the research methods literature: principles and strategies illustrated by a systematic overview of sampling in qualitative research

  • Stephen J. Gentles 1 , 4 ,
  • Cathy Charles 1 ,
  • David B. Nicholas 2 ,
  • Jenny Ploeg 3 &
  • K. Ann McKibbon 1  

Systematic Reviews volume  5 , Article number:  172 ( 2016 ) Cite this article

49k Accesses

26 Citations

13 Altmetric

Metrics details

Overviews of methods are potentially useful means to increase clarity and enhance collective understanding of specific methods topics that may be characterized by ambiguity, inconsistency, or a lack of comprehensiveness. This type of review represents a distinct literature synthesis method, although to date, its methodology remains relatively undeveloped despite several aspects that demand unique review procedures. The purpose of this paper is to initiate discussion about what a rigorous systematic approach to reviews of methods, referred to here as systematic methods overviews , might look like by providing tentative suggestions for approaching specific challenges likely to be encountered. The guidance offered here was derived from experience conducting a systematic methods overview on the topic of sampling in qualitative research.

The guidance is organized into several principles that highlight specific objectives for this type of review given the common challenges that must be overcome to achieve them. Optional strategies for achieving each principle are also proposed, along with discussion of how they were successfully implemented in the overview on sampling. We describe seven paired principles and strategies that address the following aspects: delimiting the initial set of publications to consider, searching beyond standard bibliographic databases, searching without the availability of relevant metadata, selecting publications on purposeful conceptual grounds, defining concepts and other information to abstract iteratively, accounting for inconsistent terminology used to describe specific methods topics, and generating rigorous verifiable analytic interpretations. Since a broad aim in systematic methods overviews is to describe and interpret the relevant literature in qualitative terms, we suggest that iterative decision making at various stages of the review process, and a rigorous qualitative approach to analysis are necessary features of this review type.

Conclusions

We believe that the principles and strategies provided here will be useful to anyone choosing to undertake a systematic methods overview. This paper represents an initial effort to promote high quality critical evaluations of the literature regarding problematic methods topics, which have the potential to promote clearer, shared understandings, and accelerate advances in research methods. Further work is warranted to develop more definitive guidance.

Peer Review reports

While reviews of methods are not new, they represent a distinct review type whose methodology remains relatively under-addressed in the literature despite the clear implications for unique review procedures. One of few examples to describe it is a chapter containing reflections of two contributing authors in a book of 21 reviews on methodological topics compiled for the British National Health Service, Health Technology Assessment Program [ 1 ]. Notable is their observation of how the differences between the methods reviews and conventional quantitative systematic reviews, specifically attributable to their varying content and purpose, have implications for defining what qualifies as systematic. While the authors describe general aspects of “systematicity” (including rigorous application of a methodical search, abstraction, and analysis), they also describe a high degree of variation within the category of methods reviews itself and so offer little in the way of concrete guidance. In this paper, we present tentative concrete guidance, in the form of a preliminary set of proposed principles and optional strategies, for a rigorous systematic approach to reviewing and evaluating the literature on quantitative or qualitative methods topics. For purposes of this article, we have used the term systematic methods overview to emphasize the notion of a systematic approach to such reviews.

The conventional focus of rigorous literature reviews (i.e., review types for which systematic methods have been codified, including the various approaches to quantitative systematic reviews [ 2 – 4 ], and the numerous forms of qualitative and mixed methods literature synthesis [ 5 – 10 ]) is to synthesize empirical research findings from multiple studies. By contrast, the focus of overviews of methods, including the systematic approach we advocate, is to synthesize guidance on methods topics. The literature consulted for such reviews may include the methods literature, methods-relevant sections of empirical research reports, or both. Thus, this paper adds to previous work published in this journal—namely, recent preliminary guidance for conducting reviews of theory [ 11 ]—that has extended the application of systematic review methods to novel review types that are concerned with subject matter other than empirical research findings.

Published examples of methods overviews illustrate the varying objectives they can have. One objective is to establish methodological standards for appraisal purposes. For example, reviews of existing quality appraisal standards have been used to propose universal standards for appraising the quality of primary qualitative research [ 12 ] or evaluating qualitative research reports [ 13 ]. A second objective is to survey the methods-relevant sections of empirical research reports to establish current practices on methods use and reporting practices, which Moher and colleagues [ 14 ] recommend as a means for establishing the needs to be addressed in reporting guidelines (see, for example [ 15 , 16 ]). A third objective for a methods review is to offer clarity and enhance collective understanding regarding a specific methods topic that may be characterized by ambiguity, inconsistency, or a lack of comprehensiveness within the available methods literature. An example of this is a overview whose objective was to review the inconsistent definitions of intention-to-treat analysis (the methodologically preferred approach to analyze randomized controlled trial data) that have been offered in the methods literature and propose a solution for improving conceptual clarity [ 17 ]. Such reviews are warranted because students and researchers who must learn or apply research methods typically lack the time to systematically search, retrieve, review, and compare the available literature to develop a thorough and critical sense of the varied approaches regarding certain controversial or ambiguous methods topics.

While systematic methods overviews , as a review type, include both reviews of the methods literature and reviews of methods-relevant sections from empirical study reports, the guidance provided here is primarily applicable to reviews of the methods literature since it was derived from the experience of conducting such a review [ 18 ], described below. To our knowledge, there are no well-developed proposals on how to rigorously conduct such reviews. Such guidance would have the potential to improve the thoroughness and credibility of critical evaluations of the methods literature, which could increase their utility as a tool for generating understandings that advance research methods, both qualitative and quantitative. Our aim in this paper is thus to initiate discussion about what might constitute a rigorous approach to systematic methods overviews. While we hope to promote rigor in the conduct of systematic methods overviews wherever possible, we do not wish to suggest that all methods overviews need be conducted to the same standard. Rather, we believe that the level of rigor may need to be tailored pragmatically to the specific review objectives, which may not always justify the resource requirements of an intensive review process.

The example systematic methods overview on sampling in qualitative research

The principles and strategies we propose in this paper are derived from experience conducting a systematic methods overview on the topic of sampling in qualitative research [ 18 ]. The main objective of that methods overview was to bring clarity and deeper understanding of the prominent concepts related to sampling in qualitative research (purposeful sampling strategies, saturation, etc.). Specifically, we interpreted the available guidance, commenting on areas lacking clarity, consistency, or comprehensiveness (without proposing any recommendations on how to do sampling). This was achieved by a comparative and critical analysis of publications representing the most influential (i.e., highly cited) guidance across several methodological traditions in qualitative research.

The specific methods and procedures for the overview on sampling [ 18 ] from which our proposals are derived were developed both after soliciting initial input from local experts in qualitative research and an expert health librarian (KAM) and through ongoing careful deliberation throughout the review process. To summarize, in that review, we employed a transparent and rigorous approach to search the methods literature, selected publications for inclusion according to a purposeful and iterative process, abstracted textual data using structured abstraction forms, and analyzed (synthesized) the data using a systematic multi-step approach featuring abstraction of text, summary of information in matrices, and analytic comparisons.

For this article, we reflected on both the problems and challenges encountered at different stages of the review and our means for selecting justifiable procedures to deal with them. Several principles were then derived by considering the generic nature of these problems, while the generalizable aspects of the procedures used to address them formed the basis of optional strategies. Further details of the specific methods and procedures used in the overview on qualitative sampling are provided below to illustrate both the types of objectives and challenges that reviewers will likely need to consider and our approach to implementing each of the principles and strategies.

Organization of the guidance into principles and strategies

For the purposes of this article, principles are general statements outlining what we propose are important aims or considerations within a particular review process, given the unique objectives or challenges to be overcome with this type of review. These statements follow the general format, “considering the objective or challenge of X, we propose Y to be an important aim or consideration.” Strategies are optional and flexible approaches for implementing the previous principle outlined. Thus, generic challenges give rise to principles, which in turn give rise to strategies.

We organize the principles and strategies below into three sections corresponding to processes characteristic of most systematic literature synthesis approaches: literature identification and selection ; data abstraction from the publications selected for inclusion; and analysis , including critical appraisal and synthesis of the abstracted data. Within each section, we also describe the specific methodological decisions and procedures used in the overview on sampling in qualitative research [ 18 ] to illustrate how the principles and strategies for each review process were applied and implemented in a specific case. We expect this guidance and accompanying illustrations will be useful for anyone considering engaging in a methods overview, particularly those who may be familiar with conventional systematic review methods but may not yet appreciate some of the challenges specific to reviewing the methods literature.

Results and discussion

Literature identification and selection.

The identification and selection process includes search and retrieval of publications and the development and application of inclusion and exclusion criteria to select the publications that will be abstracted and analyzed in the final review. Literature identification and selection for overviews of the methods literature is challenging and potentially more resource-intensive than for most reviews of empirical research. This is true for several reasons that we describe below, alongside discussion of the potential solutions. Additionally, we suggest in this section how the selection procedures can be chosen to match the specific analytic approach used in methods overviews.

Delimiting a manageable set of publications

One aspect of methods overviews that can make identification and selection challenging is the fact that the universe of literature containing potentially relevant information regarding most methods-related topics is expansive and often unmanageably so. Reviewers are faced with two large categories of literature: the methods literature , where the possible publication types include journal articles, books, and book chapters; and the methods-relevant sections of empirical study reports , where the possible publication types include journal articles, monographs, books, theses, and conference proceedings. In our systematic overview of sampling in qualitative research, exhaustively searching (including retrieval and first-pass screening) all publication types across both categories of literature for information on a single methods-related topic was too burdensome to be feasible. The following proposed principle follows from the need to delimit a manageable set of literature for the review.

Principle #1:

Considering the broad universe of potentially relevant literature, we propose that an important objective early in the identification and selection stage is to delimit a manageable set of methods-relevant publications in accordance with the objectives of the methods overview.

Strategy #1:

To limit the set of methods-relevant publications that must be managed in the selection process, reviewers have the option to initially review only the methods literature, and exclude the methods-relevant sections of empirical study reports, provided this aligns with the review’s particular objectives.

We propose that reviewers are justified in choosing to select only the methods literature when the objective is to map out the range of recognized concepts relevant to a methods topic, to summarize the most authoritative or influential definitions or meanings for methods-related concepts, or to demonstrate a problematic lack of clarity regarding a widely established methods-related concept and potentially make recommendations for a preferred approach to the methods topic in question. For example, in the case of the methods overview on sampling [ 18 ], the primary aim was to define areas lacking in clarity for multiple widely established sampling-related topics. In the review on intention-to-treat in the context of missing outcome data [ 17 ], the authors identified a lack of clarity based on multiple inconsistent definitions in the literature and went on to recommend separating the issue of how to handle missing outcome data from the issue of whether an intention-to-treat analysis can be claimed.

In contrast to strategy #1, it may be appropriate to select the methods-relevant sections of empirical study reports when the objective is to illustrate how a methods concept is operationalized in research practice or reported by authors. For example, one could review all the publications in 2 years’ worth of issues of five high-impact field-related journals to answer questions about how researchers describe implementing a particular method or approach, or to quantify how consistently they define or report using it. Such reviews are often used to highlight gaps in the reporting practices regarding specific methods, which may be used to justify items to address in reporting guidelines (for example, [ 14 – 16 ]).

It is worth recognizing that other authors have advocated broader positions regarding the scope of literature to be considered in a review, expanding on our perspective. Suri [ 10 ] (who, like us, emphasizes how different sampling strategies are suitable for different literature synthesis objectives) has, for example, described a two-stage literature sampling procedure (pp. 96–97). First, reviewers use an initial approach to conduct a broad overview of the field—for reviews of methods topics, this would entail an initial review of the research methods literature. This is followed by a second more focused stage in which practical examples are purposefully selected—for methods reviews, this would involve sampling the empirical literature to illustrate key themes and variations. While this approach is seductive in its capacity to generate more in depth and interpretive analytic findings, some reviewers may consider it too resource-intensive to include the second step no matter how selective the purposeful sampling. In the overview on sampling where we stopped after the first stage [ 18 ], we discussed our selective focus on the methods literature as a limitation that left opportunities for further analysis of the literature. We explicitly recommended, for example, that theoretical sampling was a topic for which a future review of the methods sections of empirical reports was justified to answer specific questions identified in the primary review.

Ultimately, reviewers must make pragmatic decisions that balance resource considerations, combined with informed predictions about the depth and complexity of literature available on their topic, with the stated objectives of their review. The remaining principles and strategies apply primarily to overviews that include the methods literature, although some aspects may be relevant to reviews that include empirical study reports.

Searching beyond standard bibliographic databases

An important reality affecting identification and selection in overviews of the methods literature is the increased likelihood for relevant publications to be located in sources other than journal articles (which is usually not the case for overviews of empirical research, where journal articles generally represent the primary publication type). In the overview on sampling [ 18 ], out of 41 full-text publications retrieved and reviewed, only 4 were journal articles, while 37 were books or book chapters. Since many books and book chapters did not exist electronically, their full text had to be physically retrieved in hardcopy, while 11 publications were retrievable only through interlibrary loan or purchase request. The tasks associated with such retrieval are substantially more time-consuming than electronic retrieval. Since a substantial proportion of methods-related guidance may be located in publication types that are less comprehensively indexed in standard bibliographic databases, identification and retrieval thus become complicated processes.

Principle #2:

Considering that important sources of methods guidance can be located in non-journal publication types (e.g., books, book chapters) that tend to be poorly indexed in standard bibliographic databases, it is important to consider alternative search methods for identifying relevant publications to be further screened for inclusion.

Strategy #2:

To identify books, book chapters, and other non-journal publication types not thoroughly indexed in standard bibliographic databases, reviewers may choose to consult one or more of the following less standard sources: Google Scholar, publisher web sites, or expert opinion.

In the case of the overview on sampling in qualitative research [ 18 ], Google Scholar had two advantages over other standard bibliographic databases: it indexes and returns records of books and book chapters likely to contain guidance on qualitative research methods topics; and it has been validated as providing higher citation counts than ISI Web of Science (a producer of numerous bibliographic databases accessible through institutional subscription) for several non-biomedical disciplines including the social sciences where qualitative research methods are prominently used [ 19 – 21 ]. While we identified numerous useful publications by consulting experts, the author publication lists generated through Google Scholar searches were uniquely useful to identify more recent editions of methods books identified by experts.

Searching without relevant metadata

Determining what publications to select for inclusion in the overview on sampling [ 18 ] could only rarely be accomplished by reviewing the publication’s metadata. This was because for the many books and other non-journal type publications we identified as possibly relevant, the potential content of interest would be located in only a subsection of the publication. In this common scenario for reviews of the methods literature (as opposed to methods overviews that include empirical study reports), reviewers will often be unable to employ standard title, abstract, and keyword database searching or screening as a means for selecting publications.

Principle #3:

Considering that the presence of information about the topic of interest may not be indicated in the metadata for books and similar publication types, it is important to consider other means of identifying potentially useful publications for further screening.

Strategy #3:

One approach to identifying potentially useful books and similar publication types is to consider what classes of such publications (e.g., all methods manuals for a certain research approach) are likely to contain relevant content, then identify, retrieve, and review the full text of corresponding publications to determine whether they contain information on the topic of interest.

In the example of the overview on sampling in qualitative research [ 18 ], the topic of interest (sampling) was one of numerous topics covered in the general qualitative research methods manuals. Consequently, examples from this class of publications first had to be identified for retrieval according to non-keyword-dependent criteria. Thus, all methods manuals within the three research traditions reviewed (grounded theory, phenomenology, and case study) that might contain discussion of sampling were sought through Google Scholar and expert opinion, their full text obtained, and hand-searched for relevant content to determine eligibility. We used tables of contents and index sections of books to aid this hand searching.

Purposefully selecting literature on conceptual grounds

A final consideration in methods overviews relates to the type of analysis used to generate the review findings. Unlike quantitative systematic reviews where reviewers aim for accurate or unbiased quantitative estimates—something that requires identifying and selecting the literature exhaustively to obtain all relevant data available (i.e., a complete sample)—in methods overviews, reviewers must describe and interpret the relevant literature in qualitative terms to achieve review objectives. In other words, the aim in methods overviews is to seek coverage of the qualitative concepts relevant to the methods topic at hand. For example, in the overview of sampling in qualitative research [ 18 ], achieving review objectives entailed providing conceptual coverage of eight sampling-related topics that emerged as key domains. The following principle recognizes that literature sampling should therefore support generating qualitative conceptual data as the input to analysis.

Principle #4:

Since the analytic findings of a systematic methods overview are generated through qualitative description and interpretation of the literature on a specified topic, selection of the literature should be guided by a purposeful strategy designed to achieve adequate conceptual coverage (i.e., representing an appropriate degree of variation in relevant ideas) of the topic according to objectives of the review.

Strategy #4:

One strategy for choosing the purposeful approach to use in selecting the literature according to the review objectives is to consider whether those objectives imply exploring concepts either at a broad overview level, in which case combining maximum variation selection with a strategy that limits yield (e.g., critical case, politically important, or sampling for influence—described below) may be appropriate; or in depth, in which case purposeful approaches aimed at revealing innovative cases will likely be necessary.

In the methods overview on sampling, the implied scope was broad since we set out to review publications on sampling across three divergent qualitative research traditions—grounded theory, phenomenology, and case study—to facilitate making informative conceptual comparisons. Such an approach would be analogous to maximum variation sampling.

At the same time, the purpose of that review was to critically interrogate the clarity, consistency, and comprehensiveness of literature from these traditions that was “most likely to have widely influenced students’ and researchers’ ideas about sampling” (p. 1774) [ 18 ]. In other words, we explicitly set out to review and critique the most established and influential (and therefore dominant) literature, since this represents a common basis of knowledge among students and researchers seeking understanding or practical guidance on sampling in qualitative research. To achieve this objective, we purposefully sampled publications according to the criterion of influence , which we operationalized as how often an author or publication has been referenced in print or informal discourse. This second sampling approach also limited the literature we needed to consider within our broad scope review to a manageable amount.

To operationalize this strategy of sampling for influence , we sought to identify both the most influential authors within a qualitative research tradition (all of whose citations were subsequently screened) and the most influential publications on the topic of interest by non-influential authors. This involved a flexible approach that combined multiple indicators of influence to avoid the dilemma that any single indicator might provide inadequate coverage. These indicators included bibliometric data (h-index for author influence [ 22 ]; number of cites for publication influence), expert opinion, and cross-references in the literature (i.e., snowball sampling). As a final selection criterion, a publication was included only if it made an original contribution in terms of novel guidance regarding sampling or a related concept; thus, purely secondary sources were excluded. Publish or Perish software (Anne-Wil Harzing; available at http://www.harzing.com/resources/publish-or-perish ) was used to generate bibliometric data via the Google Scholar database. Figure  1 illustrates how identification and selection in the methods overview on sampling was a multi-faceted and iterative process. The authors selected as influential, and the publications selected for inclusion or exclusion are listed in Additional file 1 (Matrices 1, 2a, 2b).

Literature identification and selection process used in the methods overview on sampling [ 18 ]

In summary, the strategies of seeking maximum variation and sampling for influence were employed in the sampling overview to meet the specific review objectives described. Reviewers will need to consider the full range of purposeful literature sampling approaches at their disposal in deciding what best matches the specific aims of their own reviews. Suri [ 10 ] has recently retooled Patton’s well-known typology of purposeful sampling strategies (originally intended for primary research) for application to literature synthesis, providing a useful resource in this respect.

Data abstraction

The purpose of data abstraction in rigorous literature reviews is to locate and record all data relevant to the topic of interest from the full text of included publications, making them available for subsequent analysis. Conventionally, a data abstraction form—consisting of numerous distinct conceptually defined fields to which corresponding information from the source publication is recorded—is developed and employed. There are several challenges, however, to the processes of developing the abstraction form and abstracting the data itself when conducting methods overviews, which we address here. Some of these problems and their solutions may be familiar to those who have conducted qualitative literature syntheses, which are similarly conceptual.

Iteratively defining conceptual information to abstract

In the overview on sampling [ 18 ], while we surveyed multiple sources beforehand to develop a list of concepts relevant for abstraction (e.g., purposeful sampling strategies, saturation, sample size), there was no way for us to anticipate some concepts prior to encountering them in the review process. Indeed, in many cases, reviewers are unable to determine the complete set of methods-related concepts that will be the focus of the final review a priori without having systematically reviewed the publications to be included. Thus, defining what information to abstract beforehand may not be feasible.

Principle #5:

Considering the potential impracticality of defining a complete set of relevant methods-related concepts from a body of literature one has not yet systematically read, selecting and defining fields for data abstraction must often be undertaken iteratively. Thus, concepts to be abstracted can be expected to grow and change as data abstraction proceeds.

Strategy #5:

Reviewers can develop an initial form or set of concepts for abstraction purposes according to standard methods (e.g., incorporating expert feedback, pilot testing) and remain attentive to the need to iteratively revise it as concepts are added or modified during the review. Reviewers should document revisions and return to re-abstract data from previously abstracted publications as the new data requirements are determined.

In the sampling overview [ 18 ], we developed and maintained the abstraction form in Microsoft Word. We derived the initial set of abstraction fields from our own knowledge of relevant sampling-related concepts, consultation with local experts, and reviewing a pilot sample of publications. Since the publications in this review included a large proportion of books, the abstraction process often began by flagging the broad sections within a publication containing topic-relevant information for detailed review to identify text to abstract. When reviewing flagged text, the reviewer occasionally encountered an unanticipated concept significant enough to warrant being added as a new field to the abstraction form. For example, a field was added to capture how authors described the timing of sampling decisions, whether before (a priori) or after (ongoing) starting data collection, or whether this was unclear. In these cases, we systematically documented the modification to the form and returned to previously abstracted publications to abstract any information that might be relevant to the new field.

The logic of this strategy is analogous to the logic used in a form of research synthesis called best fit framework synthesis (BFFS) [ 23 – 25 ]. In that method, reviewers initially code evidence using an a priori framework they have selected. When evidence cannot be accommodated by the selected framework, reviewers then develop new themes or concepts from which they construct a new expanded framework. Both the strategy proposed and the BFFS approach to research synthesis are notable for their rigorous and transparent means to adapt a final set of concepts to the content under review.

Accounting for inconsistent terminology

An important complication affecting the abstraction process in methods overviews is that the language used by authors to describe methods-related concepts can easily vary across publications. For example, authors from different qualitative research traditions often use different terms for similar methods-related concepts. Furthermore, as we found in the sampling overview [ 18 ], there may be cases where no identifiable term, phrase, or label for a methods-related concept is used at all, and a description of it is given instead. This can make searching the text for relevant concepts based on keywords unreliable.

Principle #6:

Since accepted terms may not be used consistently to refer to methods concepts, it is necessary to rely on the definitions for concepts, rather than keywords, to identify relevant information in the publication to abstract.

Strategy #6:

An effective means to systematically identify relevant information is to develop and iteratively adjust written definitions for key concepts (corresponding to abstraction fields) that are consistent with and as inclusive of as much of the literature reviewed as possible. Reviewers then seek information that matches these definitions (rather than keywords) when scanning a publication for relevant data to abstract.

In the abstraction process for the sampling overview [ 18 ], we noted the several concepts of interest to the review for which abstraction by keyword was particularly problematic due to inconsistent terminology across publications: sampling , purposeful sampling , sampling strategy , and saturation (for examples, see Additional file 1 , Matrices 3a, 3b, 4). We iteratively developed definitions for these concepts by abstracting text from publications that either provided an explicit definition or from which an implicit definition could be derived, which was recorded in fields dedicated to the concept’s definition. Using a method of constant comparison, we used text from definition fields to inform and modify a centrally maintained definition of the corresponding concept to optimize its fit and inclusiveness with the literature reviewed. Table  1 shows, as an example, the final definition constructed in this way for one of the central concepts of the review, qualitative sampling .

We applied iteratively developed definitions when making decisions about what specific text to abstract for an existing field, which allowed us to abstract concept-relevant data even if no recognized keyword was used. For example, this was the case for the sampling-related concept, saturation , where the relevant text available for abstraction in one publication [ 26 ]—“to continue to collect data until nothing new was being observed or recorded, no matter how long that takes”—was not accompanied by any term or label whatsoever.

This comparative analytic strategy (and our approach to analysis more broadly as described in strategy #7, below) is analogous to the process of reciprocal translation —a technique first introduced for meta-ethnography by Noblit and Hare [ 27 ] that has since been recognized as a common element in a variety of qualitative metasynthesis approaches [ 28 ]. Reciprocal translation, taken broadly, involves making sense of a study’s findings in terms of the findings of the other studies included in the review. In practice, it has been operationalized in different ways. Melendez-Torres and colleagues developed a typology from their review of the metasynthesis literature, describing four overlapping categories of specific operations undertaken in reciprocal translation: visual representation, key paper integration, data reduction and thematic extraction, and line-by-line coding [ 28 ]. The approaches suggested in both strategies #6 and #7, with their emphasis on constant comparison, appear to fall within the line-by-line coding category.

Generating credible and verifiable analytic interpretations

The analysis in a systematic methods overview must support its more general objective, which we suggested above is often to offer clarity and enhance collective understanding regarding a chosen methods topic. In our experience, this involves describing and interpreting the relevant literature in qualitative terms. Furthermore, any interpretative analysis required may entail reaching different levels of abstraction, depending on the more specific objectives of the review. For example, in the overview on sampling [ 18 ], we aimed to produce a comparative analysis of how multiple sampling-related topics were treated differently within and among different qualitative research traditions. To promote credibility of the review, however, not only should one seek a qualitative analytic approach that facilitates reaching varying levels of abstraction but that approach must also ensure that abstract interpretations are supported and justified by the source data and not solely the product of the analyst’s speculative thinking.

Principle #7:

Considering the qualitative nature of the analysis required in systematic methods overviews, it is important to select an analytic method whose interpretations can be verified as being consistent with the literature selected, regardless of the level of abstraction reached.

Strategy #7:

We suggest employing the constant comparative method of analysis [ 29 ] because it supports developing and verifying analytic links to the source data throughout progressively interpretive or abstract levels. In applying this approach, we advise a rigorous approach, documenting how supportive quotes or references to the original texts are carried forward in the successive steps of analysis to allow for easy verification.

The analytic approach used in the methods overview on sampling [ 18 ] comprised four explicit steps, progressing in level of abstraction—data abstraction, matrices, narrative summaries, and final analytic conclusions (Fig.  2 ). While we have positioned data abstraction as the second stage of the generic review process (prior to Analysis), above, we also considered it as an initial step of analysis in the sampling overview for several reasons. First, it involved a process of constant comparisons and iterative decision-making about the fields to add or define during development and modification of the abstraction form, through which we established the range of concepts to be addressed in the review. At the same time, abstraction involved continuous analytic decisions about what textual quotes (ranging in size from short phrases to numerous paragraphs) to record in the fields thus created. This constant comparative process was analogous to open coding in which textual data from publications was compared to conceptual fields (equivalent to codes) or to other instances of data previously abstracted when constructing definitions to optimize their fit with the overall literature as described in strategy #6. Finally, in the data abstraction step, we also recorded our first interpretive thoughts in dedicated fields, providing initial material for the more abstract analytic steps.

Summary of progressive steps of analysis used in the methods overview on sampling [ 18 ]

In the second step of the analysis, we constructed topic-specific matrices , or tables, by copying relevant quotes from abstraction forms into the appropriate cells of matrices (for the complete set of analytic matrices developed in the sampling review, see Additional file 1 (matrices 3 to 10)). Each matrix ranged from one to five pages; row headings, nested three-deep, identified the methodological tradition, author, and publication, respectively; and column headings identified the concepts, which corresponded to abstraction fields. Matrices thus allowed us to make further comparisons across methodological traditions, and between authors within a tradition. In the third step of analysis, we recorded our comparative observations as narrative summaries , in which we used illustrative quotes more sparingly. In the final step, we developed analytic conclusions based on the narrative summaries about the sampling-related concepts within each methodological tradition for which clarity, consistency, or comprehensiveness of the available guidance appeared to be lacking. Higher levels of analysis thus built logically from the lower levels, enabling us to easily verify analytic conclusions by tracing the support for claims by comparing the original text of publications reviewed.

Integrative versus interpretive methods overviews

The analytic product of systematic methods overviews is comparable to qualitative evidence syntheses, since both involve describing and interpreting the relevant literature in qualitative terms. Most qualitative synthesis approaches strive to produce new conceptual understandings that vary in level of interpretation. Dixon-Woods and colleagues [ 30 ] elaborate on a useful distinction, originating from Noblit and Hare [ 27 ], between integrative and interpretive reviews. Integrative reviews focus on summarizing available primary data and involve using largely secure and well defined concepts to do so; definitions are used from an early stage to specify categories for abstraction (or coding) of data, which in turn supports their aggregation; they do not seek as their primary focus to develop or specify new concepts, although they may achieve some theoretical or interpretive functions. For interpretive reviews, meanwhile, the main focus is to develop new concepts and theories that integrate them, with the implication that the concepts developed become fully defined towards the end of the analysis. These two forms are not completely distinct, and “every integrative synthesis will include elements of interpretation, and every interpretive synthesis will include elements of aggregation of data” [ 30 ].

The example methods overview on sampling [ 18 ] could be classified as predominantly integrative because its primary goal was to aggregate influential authors’ ideas on sampling-related concepts; there were also, however, elements of interpretive synthesis since it aimed to develop new ideas about where clarity in guidance on certain sampling-related topics is lacking, and definitions for some concepts were flexible and not fixed until late in the review. We suggest that most systematic methods overviews will be classifiable as predominantly integrative (aggregative). Nevertheless, more highly interpretive methods overviews are also quite possible—for example, when the review objective is to provide a highly critical analysis for the purpose of generating new methodological guidance. In such cases, reviewers may need to sample more deeply (see strategy #4), specifically by selecting empirical research reports (i.e., to go beyond dominant or influential ideas in the methods literature) that are likely to feature innovations or instructive lessons in employing a given method.

In this paper, we have outlined tentative guidance in the form of seven principles and strategies on how to conduct systematic methods overviews, a review type in which methods-relevant literature is systematically analyzed with the aim of offering clarity and enhancing collective understanding regarding a specific methods topic. Our proposals include strategies for delimiting the set of publications to consider, searching beyond standard bibliographic databases, searching without the availability of relevant metadata, selecting publications on purposeful conceptual grounds, defining concepts and other information to abstract iteratively, accounting for inconsistent terminology, and generating credible and verifiable analytic interpretations. We hope the suggestions proposed will be useful to others undertaking reviews on methods topics in future.

As far as we are aware, this is the first published source of concrete guidance for conducting this type of review. It is important to note that our primary objective was to initiate methodological discussion by stimulating reflection on what rigorous methods for this type of review should look like, leaving the development of more complete guidance to future work. While derived from the experience of reviewing a single qualitative methods topic, we believe the principles and strategies provided are generalizable to overviews of both qualitative and quantitative methods topics alike. However, it is expected that additional challenges and insights for conducting such reviews have yet to be defined. Thus, we propose that next steps for developing more definitive guidance should involve an attempt to collect and integrate other reviewers’ perspectives and experiences in conducting systematic methods overviews on a broad range of qualitative and quantitative methods topics. Formalized guidance and standards would improve the quality of future methods overviews, something we believe has important implications for advancing qualitative and quantitative methodology. When undertaken to a high standard, rigorous critical evaluations of the available methods guidance have significant potential to make implicit controversies explicit, and improve the clarity and precision of our understandings of problematic qualitative or quantitative methods issues.

A review process central to most types of rigorous reviews of empirical studies, which we did not explicitly address in a separate review step above, is quality appraisal . The reason we have not treated this as a separate step stems from the different objectives of the primary publications included in overviews of the methods literature (i.e., providing methodological guidance) compared to the primary publications included in the other established review types (i.e., reporting findings from single empirical studies). This is not to say that appraising quality of the methods literature is not an important concern for systematic methods overviews. Rather, appraisal is much more integral to (and difficult to separate from) the analysis step, in which we advocate appraising clarity, consistency, and comprehensiveness—the quality appraisal criteria that we suggest are appropriate for the methods literature. As a second important difference regarding appraisal, we currently advocate appraising the aforementioned aspects at the level of the literature in aggregate rather than at the level of individual publications. One reason for this is that methods guidance from individual publications generally builds on previous literature, and thus we feel that ahistorical judgments about comprehensiveness of single publications lack relevance and utility. Additionally, while different methods authors may express themselves less clearly than others, their guidance can nonetheless be highly influential and useful, and should therefore not be downgraded or ignored based on considerations of clarity—which raises questions about the alternative uses that quality appraisals of individual publications might have. Finally, legitimate variability in the perspectives that methods authors wish to emphasize, and the levels of generality at which they write about methods, makes critiquing individual publications based on the criterion of clarity a complex and potentially problematic endeavor that is beyond the scope of this paper to address. By appraising the current state of the literature at a holistic level, reviewers stand to identify important gaps in understanding that represent valuable opportunities for further methodological development.

To summarize, the principles and strategies provided here may be useful to those seeking to undertake their own systematic methods overview. Additional work is needed, however, to establish guidance that is comprehensive by comparing the experiences from conducting a variety of methods overviews on a range of methods topics. Efforts that further advance standards for systematic methods overviews have the potential to promote high-quality critical evaluations that produce conceptually clear and unified understandings of problematic methods topics, thereby accelerating the advance of research methodology.

Hutton JL, Ashcroft R. What does “systematic” mean for reviews of methods? In: Black N, Brazier J, Fitzpatrick R, Reeves B, editors. Health services research methods: a guide to best practice. London: BMJ Publishing Group; 1998. p. 249–54.

Google Scholar  

Cochrane handbook for systematic reviews of interventions. In. Edited by Higgins JPT, Green S, Version 5.1.0 edn: The Cochrane Collaboration; 2011.

Centre for Reviews and Dissemination: Systematic reviews: CRD’s guidance for undertaking reviews in health care . York: Centre for Reviews and Dissemination; 2009.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700–0.

Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009;9(1):59.

Article   PubMed   PubMed Central   Google Scholar  

Kastner M, Tricco AC, Soobiah C, Lillie E, Perrier L, Horsley T, Welch V, Cogo E, Antony J, Straus SE. What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Med Res Methodol. 2012;12(1):1–1.

Article   Google Scholar  

Booth A, Noyes J, Flemming K, Gerhardus A. Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions. In: Integrate-HTA. 2016.

Booth A, Sutton A, Papaioannou D. Systematic approaches to successful literature review. 2nd ed. London: Sage; 2016.

Hannes K, Lockwood C. Synthesizing qualitative research: choosing the right approach. Chichester: Wiley-Blackwell; 2012.

Suri H. Towards methodologically inclusive research syntheses: expanding possibilities. New York: Routledge; 2014.

Campbell M, Egan M, Lorenc T, Bond L, Popham F, Fenton C, Benzeval M. Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health. Syst Rev. 2014;3(1):1–11.

Cohen DJ, Crabtree BF. Evaluative criteria for qualitative research in health care: controversies and recommendations. Ann Fam Med. 2008;6(4):331–9.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reportingqualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Article   PubMed   Google Scholar  

Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2):e1000217.

Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4(3):e78.

Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet. 2005;365(9465):1159–62.

Alshurafa M, Briel M, Akl EA, Haines T, Moayyedi P, Gentles SJ, Rios L, Tran C, Bhatnagar N, Lamontagne F, et al. Inconsistent definitions for intention-to-treat in relation to missing outcome data: systematic review of the methods literature. PLoS One. 2012;7(11):e49163.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gentles SJ, Charles C, Ploeg J, McKibbon KA. Sampling in qualitative research: insights from an overview of the methods literature. Qual Rep. 2015;20(11):1772–89.

Harzing A-W, Alakangas S. Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison. Scientometrics. 2016;106(2):787–804.

Harzing A-WK, van der Wal R. Google Scholar as a new source for citation analysis. Ethics Sci Environ Polit. 2008;8(1):61–73.

Kousha K, Thelwall M. Google Scholar citations and Google Web/URL citations: a multi‐discipline exploratory analysis. J Assoc Inf Sci Technol. 2007;58(7):1055–65.

Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569–72.

Booth A, Carroll C. How to build up the actionable knowledge base: the role of ‘best fit’ framework synthesis for studies of improvement in healthcare. BMJ Quality Safety. 2015;24(11):700–8.

Carroll C, Booth A, Leaviss J, Rick J. “Best fit” framework synthesis: refining the method. BMC Med Res Methodol. 2013;13(1):37.

Carroll C, Booth A, Cooper K. A worked example of “best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents. BMC Med Res Methodol. 2011;11(1):29.

Cohen MZ, Kahn DL, Steeves DL. Hermeneutic phenomenological research: a practical guide for nurse researchers. Thousand Oaks: Sage; 2000.

Noblit GW, Hare RD. Meta-ethnography: synthesizing qualitative studies. Newbury Park: Sage; 1988.

Book   Google Scholar  

Melendez-Torres GJ, Grant S, Bonell C. A systematic review and critical appraisal of qualitative metasynthetic practice in public health to develop a taxonomy of operations of reciprocal translation. Res Synthesis Methods. 2015;6(4):357–71.

Article   CAS   Google Scholar  

Glaser BG, Strauss A. The discovery of grounded theory. Chicago: Aldine; 1967.

Dixon-Woods M, Agarwal S, Young B, Jones D, Sutton A. Integrative approaches to qualitative and quantitative evidence. In: UK National Health Service. 2004. p. 1–44.

Download references

Acknowledgements

Not applicable.

There was no funding for this work.

Availability of data and materials

The systematic methods overview used as a worked example in this article (Gentles SJ, Charles C, Ploeg J, McKibbon KA: Sampling in qualitative research: insights from an overview of the methods literature. The Qual Rep 2015, 20(11):1772-1789) is available from http://nsuworks.nova.edu/tqr/vol20/iss11/5 .

Authors’ contributions

SJG wrote the first draft of this article, with CC contributing to drafting. All authors contributed to revising the manuscript. All authors except CC (deceased) approved the final draft. SJG, CC, KAB, and JP were involved in developing methods for the systematic methods overview on sampling.

Authors’ information

Competing interests.

The authors declare that they have no competing interests.

Consent for publication

Ethics approval and consent to participate, author information, authors and affiliations.

Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada

Stephen J. Gentles, Cathy Charles & K. Ann McKibbon

Faculty of Social Work, University of Calgary, Alberta, Canada

David B. Nicholas

School of Nursing, McMaster University, Hamilton, Ontario, Canada

Jenny Ploeg

CanChild Centre for Childhood Disability Research, McMaster University, 1400 Main Street West, IAHS 408, Hamilton, ON, L8S 1C7, Canada

Stephen J. Gentles

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Stephen J. Gentles .

Additional information

Cathy Charles is deceased

Additional file

Additional file 1:.

Submitted: Analysis_matrices. (DOC 330 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Gentles, S.J., Charles, C., Nicholas, D.B. et al. Reviewing the research methods literature: principles and strategies illustrated by a systematic overview of sampling in qualitative research. Syst Rev 5 , 172 (2016). https://doi.org/10.1186/s13643-016-0343-0

Download citation

Received : 06 June 2016

Accepted : 14 September 2016

Published : 11 October 2016

DOI : https://doi.org/10.1186/s13643-016-0343-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Systematic review
  • Literature selection
  • Research methods
  • Research methodology
  • Overview of methods
  • Systematic methods overview
  • Review methods

Systematic Reviews

ISSN: 2046-4053

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

research literature literature

University of Texas

  • University of Texas Libraries

Literature Reviews

  • What is a literature review?
  • Steps in the Literature Review Process
  • Define your research question
  • Determine inclusion and exclusion criteria
  • Choose databases and search
  • Review Results
  • Synthesize Results
  • Analyze Results
  • Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

  • Provide a comprehensive and updated review of the literature;
  • Explain why this review has taken place;
  • Articulate a position or hypothesis;
  • Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

  • Demonstrate how a study fills a gap in research
  • Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

  • Organizes or describes a topic
  • Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

  • It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
  • It may be too focused. Some niche studies may miss the bigger picture.
  • It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
  • It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy ​

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

  • October 26, 2022 recording
  • Last Updated: Oct 26, 2022 2:49 PM
  • URL: https://guides.lib.utexas.edu/literaturereviews

Creative Commons License

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base
  • Starting the research process
  • How to Write a Research Proposal | Examples & Templates

How to Write a Research Proposal | Examples & Templates

Published on October 12, 2022 by Shona McCombes and Tegan George. Revised on November 21, 2023.

Structure of a research proposal

A research proposal describes what you will investigate, why it’s important, and how you will conduct your research.

The format of a research proposal varies between fields, but most proposals will contain at least these elements:

Introduction

Literature review.

  • Research design

Reference list

While the sections may vary, the overall objective is always the same. A research proposal serves as a blueprint and guide for your research plan, helping you get organized and feel confident in the path forward you choose to take.

Table of contents

Research proposal purpose, research proposal examples, research design and methods, contribution to knowledge, research schedule, other interesting articles, frequently asked questions about research proposals.

Academics often have to write research proposals to get funding for their projects. As a student, you might have to write a research proposal as part of a grad school application , or prior to starting your thesis or dissertation .

In addition to helping you figure out what your research can look like, a proposal can also serve to demonstrate why your project is worth pursuing to a funder, educational institution, or supervisor.

Research proposal length

The length of a research proposal can vary quite a bit. A bachelor’s or master’s thesis proposal can be just a few pages, while proposals for PhD dissertations or research funding are usually much longer and more detailed. Your supervisor can help you determine the best length for your work.

One trick to get started is to think of your proposal’s structure as a shorter version of your thesis or dissertation , only without the results , conclusion and discussion sections.

Download our research proposal template

Prevent plagiarism. Run a free check.

Writing a research proposal can be quite challenging, but a good starting point could be to look at some examples. We’ve included a few for you below.

  • Example research proposal #1: “A Conceptual Framework for Scheduling Constraint Management”
  • Example research proposal #2: “Medical Students as Mediators of Change in Tobacco Use”

Like your dissertation or thesis, the proposal will usually have a title page that includes:

  • The proposed title of your project
  • Your supervisor’s name
  • Your institution and department

The first part of your proposal is the initial pitch for your project. Make sure it succinctly explains what you want to do and why.

Your introduction should:

  • Introduce your topic
  • Give necessary background and context
  • Outline your  problem statement  and research questions

To guide your introduction , include information about:

  • Who could have an interest in the topic (e.g., scientists, policymakers)
  • How much is already known about the topic
  • What is missing from this current knowledge
  • What new insights your research will contribute
  • Why you believe this research is worth doing

As you get started, it’s important to demonstrate that you’re familiar with the most important research on your topic. A strong literature review  shows your reader that your project has a solid foundation in existing knowledge or theory. It also shows that you’re not simply repeating what other people have already done or said, but rather using existing research as a jumping-off point for your own.

In this section, share exactly how your project will contribute to ongoing conversations in the field by:

  • Comparing and contrasting the main theories, methods, and debates
  • Examining the strengths and weaknesses of different approaches
  • Explaining how will you build on, challenge, or synthesize prior scholarship

Following the literature review, restate your main  objectives . This brings the focus back to your own project. Next, your research design or methodology section will describe your overall approach, and the practical steps you will take to answer your research questions.

To finish your proposal on a strong note, explore the potential implications of your research for your field. Emphasize again what you aim to contribute and why it matters.

For example, your results might have implications for:

  • Improving best practices
  • Informing policymaking decisions
  • Strengthening a theory or model
  • Challenging popular or scientific beliefs
  • Creating a basis for future research

Last but not least, your research proposal must include correct citations for every source you have used, compiled in a reference list . To create citations quickly and easily, you can use our free APA citation generator .

Some institutions or funders require a detailed timeline of the project, asking you to forecast what you will do at each stage and how long it may take. While not always required, be sure to check the requirements of your project.

Here’s an example schedule to help you get started. You can also download a template at the button below.

Download our research schedule template

If you are applying for research funding, chances are you will have to include a detailed budget. This shows your estimates of how much each part of your project will cost.

Make sure to check what type of costs the funding body will agree to cover. For each item, include:

  • Cost : exactly how much money do you need?
  • Justification : why is this cost necessary to complete the research?
  • Source : how did you calculate the amount?

To determine your budget, think about:

  • Travel costs : do you need to go somewhere to collect your data? How will you get there, and how much time will you need? What will you do there (e.g., interviews, archival research)?
  • Materials : do you need access to any tools or technologies?
  • Help : do you need to hire any research assistants for the project? What will they do, and how much will you pay them?

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

Methodology

  • Sampling methods
  • Simple random sampling
  • Stratified sampling
  • Cluster sampling
  • Likert scales
  • Reproducibility

 Statistics

  • Null hypothesis
  • Statistical power
  • Probability distribution
  • Effect size
  • Poisson distribution

Research bias

  • Optimism bias
  • Cognitive bias
  • Implicit bias
  • Hawthorne effect
  • Anchoring bias
  • Explicit bias

Once you’ve decided on your research objectives , you need to explain them in your paper, at the end of your problem statement .

Keep your research objectives clear and concise, and use appropriate verbs to accurately convey the work that you will carry out for each one.

I will compare …

A research aim is a broad statement indicating the general purpose of your research project. It should appear in your introduction at the end of your problem statement , before your research objectives.

Research objectives are more specific than your research aim. They indicate the specific ways you’ll address the overarching aim.

A PhD, which is short for philosophiae doctor (doctor of philosophy in Latin), is the highest university degree that can be obtained. In a PhD, students spend 3–5 years writing a dissertation , which aims to make a significant, original contribution to current knowledge.

A PhD is intended to prepare students for a career as a researcher, whether that be in academia, the public sector, or the private sector.

A master’s is a 1- or 2-year graduate degree that can prepare you for a variety of careers.

All master’s involve graduate-level coursework. Some are research-intensive and intend to prepare students for further study in a PhD; these usually require their students to write a master’s thesis . Others focus on professional training for a specific career.

Critical thinking refers to the ability to evaluate information and to be aware of biases or assumptions, including your own.

Like information literacy , it involves evaluating arguments, identifying and solving problems in an objective and systematic way, and clearly communicating your ideas.

The best way to remember the difference between a research plan and a research proposal is that they have fundamentally different audiences. A research plan helps you, the researcher, organize your thoughts. On the other hand, a dissertation proposal or research proposal aims to convince others (e.g., a supervisor, a funding body, or a dissertation committee) that your research topic is relevant and worthy of being conducted.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. & George, T. (2023, November 21). How to Write a Research Proposal | Examples & Templates. Scribbr. Retrieved February 16, 2024, from https://www.scribbr.com/research-process/research-proposal/

Is this article helpful?

Shona McCombes

Shona McCombes

Other students also liked, how to write a problem statement | guide & examples, writing strong research questions | criteria & examples, how to write a literature review | guide, examples, & templates, what is your plagiarism score.

  • Affiliate Program

Wordvice

  • UNITED STATES
  • 台灣 (TAIWAN)
  • TÜRKIYE (TURKEY)
  • Academic Editing Services
  • - Research Paper
  • - Journal Manuscript
  • - Dissertation
  • - College & University Assignments
  • Admissions Editing Services
  • - Application Essay
  • - Personal Statement
  • - Recommendation Letter
  • - Cover Letter
  • - CV/Resume
  • Business Editing Services
  • - Business Documents
  • - Report & Brochure
  • - Website & Blog
  • Writer Editing Services
  • - Script & Screenplay
  • Our Editors
  • Client Reviews
  • Editing & Proofreading Prices
  • Wordvice Points
  • Partner Discount
  • Plagiarism Checker
  • APA Citation Generator
  • MLA Citation Generator
  • Chicago Citation Generator
  • Vancouver Citation Generator
  • - APA Style
  • - MLA Style
  • - Chicago Style
  • - Vancouver Style
  • Writing & Editing Guide
  • Academic Resources
  • Admissions Resources

How to Make a Literature Review in Research (RRL Example)

research literature literature

What is an RRL in a research paper?

A relevant review of the literature (RRL) is an objective, concise, critical summary of published research literature relevant to a topic being researched in an article. In an RRL, you discuss knowledge and findings from existing literature relevant to your study topic. If there are conflicts or gaps in existing literature, you can also discuss these in your review, as well as how you will confront these missing elements or resolve these issues in your study.

To complete an RRL, you first need to collect relevant literature; this can include online and offline sources. Save all of your applicable resources as you will need to include them in your paper. When looking through these sources, take notes and identify concepts of each source to describe in the review of the literature.

A good RRL does NOT:

A literature review does not simply reference and list all of the material you have cited in your paper.

  • Presenting material that is not directly relevant to your study will distract and frustrate the reader and make them lose sight of the purpose of your study.
  • Starting a literature review with “A number of scholars have studied the relationship between X and Y” and simply listing who has studied the topic and what each scholar concluded is not going to strengthen your paper.

A good RRL DOES:

  • Present a brief typology that orders articles and books into groups to help readers focus on unresolved debates, inconsistencies, tensions, and new questions about a research topic.
  • Summarize the most relevant and important aspects of the scientific literature related to your area of research
  • Synthesize what has been done in this area of research and by whom, highlight what previous research indicates about a topic, and identify potential gaps and areas of disagreement in the field
  • Give the reader an understanding of the background of the field and show which studies are important—and highlight errors in previous studies

How long is a review of the literature for a research paper?

The length of a review of the literature depends on its purpose and target readership and can vary significantly in scope and depth. In a dissertation, thesis, or standalone review of literature, it is usually a full chapter of the text (at least 20 pages). Whereas, a standard research article or school assignment literature review section could only be a few paragraphs in the Introduction section .

Building Your Literature Review Bookshelf

One way to conceive of a literature review is to think about writing it as you would build a bookshelf. You don’t need to cut each piece by yourself from scratch. Rather, you can take the pieces that other researchers have cut out and put them together to build a framework on which to hang your own “books”—that is, your own study methods, results, and conclusions.

literature review bookshelf

What Makes a Good Literature Review?

The contents of a literature review (RRL) are determined by many factors, including its precise purpose in the article, the degree of consensus with a given theory or tension between competing theories, the length of the article, the number of previous studies existing in the given field, etc. The following are some of the most important elements that a literature review provides.

Historical background for your research

Analyze what has been written about your field of research to highlight what is new and significant in your study—or how the analysis itself contributes to the understanding of this field, even in a small way. Providing a historical background also demonstrates to other researchers and journal editors your competency in discussing theoretical concepts. You should also make sure to understand how to paraphrase scientific literature to avoid plagiarism in your work.

The current context of your research

Discuss central (or peripheral) questions, issues, and debates in the field. Because a field is constantly being updated by new work, you can show where your research fits into this context and explain developments and trends in research.

A discussion of relevant theories and concepts

Theories and concepts should provide the foundation for your research. For example, if you are researching the relationship between ecological environments and human populations, provide models and theories that focus on specific aspects of this connection to contextualize your study. If your study asks a question concerning sustainability, mention a theory or model that underpins this concept. If it concerns invasive species, choose material that is focused in this direction.

Definitions of relevant terminology

In the natural sciences, the meaning of terms is relatively straightforward and consistent. But if you present a term that is obscure or context-specific, you should define the meaning of the term in the Introduction section (if you are introducing a study) or in the summary of the literature being reviewed.

Description of related relevant research

Include a description of related research that shows how your work expands or challenges earlier studies or fills in gaps in previous work. You can use your literature review as evidence of what works, what doesn’t, and what is missing in the field.

Supporting evidence for a practical problem or issue your research is addressing that demonstrates its importance: Referencing related research establishes your area of research as reputable and shows you are building upon previous work that other researchers have deemed significant.

Types of Literature Reviews

Literature reviews can differ in structure, length, amount, and breadth of content included. They can range from selective (a very narrow area of research or only a single work) to comprehensive (a larger amount or range of works). They can also be part of a larger work or stand on their own.

types of literature reviews

  • A course assignment is an example of a selective, stand-alone work. It focuses on a small segment of the literature on a topic and makes up an entire work on its own.
  • The literature review in a dissertation or thesis is both comprehensive and helps make up a larger work.
  • A majority of journal articles start with a selective literature review to provide context for the research reported in the study; such a literature review is usually included in the Introduction section (but it can also follow the presentation of the results in the Discussion section ).
  • Some literature reviews are both comprehensive and stand as a separate work—in this case, the entire article analyzes the literature on a given topic.

Literature Reviews Found in Academic Journals

The two types of literature reviews commonly found in journals are those introducing research articles (studies and surveys) and stand-alone literature analyses. They can differ in their scope, length, and specific purpose.

Literature reviews introducing research articles

The literature review found at the beginning of a journal article is used to introduce research related to the specific study and is found in the Introduction section, usually near the end. It is shorter than a stand-alone review because it must be limited to very specific studies and theories that are directly relevant to the current study. Its purpose is to set research precedence and provide support for the study’s theory, methods, results, and/or conclusions. Not all research articles contain an explicit review of the literature, but most do, whether it is a discrete section or indistinguishable from the rest of the Introduction.

How to structure a literature review for an article

When writing a literature review as part of an introduction to a study, simply follow the structure of the Introduction and move from the general to the specific—presenting the broadest background information about a topic first and then moving to specific studies that support your rationale , finally leading to your hypothesis statement. Such a literature review is often indistinguishable from the Introduction itself—the literature is INTRODUCING the background and defining the gaps your study aims to fill.

The stand-alone literature review

The literature review published as a stand-alone article presents and analyzes as many of the important publications in an area of study as possible to provide background information and context for a current area of research or a study. Stand-alone reviews are an excellent resource for researchers when they are first searching for the most relevant information on an area of study.

Such literature reviews are generally a bit broader in scope and can extend further back in time. This means that sometimes a scientific literature review can be highly theoretical, in addition to focusing on specific methods and outcomes of previous studies. In addition, all sections of such a “review article” refer to existing literature rather than describing the results of the authors’ own study.

In addition, this type of literature review is usually much longer than the literature review introducing a study. At the end of the review follows a conclusion that once again explicitly ties all of the cited works together to show how this analysis is itself a contribution to the literature. While not absolutely necessary, such articles often include the terms “Literature Review” or “Review of the Literature” in the title. Whether or not that is necessary or appropriate can also depend on the specific author instructions of the target journal. Have a look at this article for more input on how to compile a stand-alone review article that is insightful and helpful for other researchers in your field.

literature review examples

How to Write a Literature Review in 6 Steps

So how do authors turn a network of articles into a coherent review of relevant literature?

Writing a literature review is not usually a linear process—authors often go back and check the literature while reformulating their ideas or making adjustments to their study. Sometimes new findings are published before a study is completed and need to be incorporated into the current work. This also means you will not be writing the literature review at any one time, but constantly working on it before, during, and after your study is complete.

Here are some steps that will help you begin and follow through on your literature review.

Step 1: Choose a topic to write about—focus on and explore this topic.

Choose a topic that you are familiar with and highly interested in analyzing; a topic your intended readers and researchers will find interesting and useful; and a topic that is current, well-established in the field, and about which there has been sufficient research conducted for a review. This will help you find the “sweet spot” for what to focus on.

Step 2: Research and collect all the scholarly information on the topic that might be pertinent to your study.

This includes scholarly articles, books, conventions, conferences, dissertations, and theses—these and any other academic work related to your area of study is called “the literature.”

Step 3: Analyze the network of information that extends or responds to the major works in your area; select the material that is most useful.

Use thought maps and charts to identify intersections in the research and to outline important categories; select the material that will be most useful to your review.

Step 4: Describe and summarize each article—provide the essential information of the article that pertains to your study.

Determine 2-3 important concepts (depending on the length of your article) that are discussed in the literature; take notes about all of the important aspects of this study relevant to the topic being reviewed.

For example, in a given study, perhaps some of the main concepts are X, Y, and Z. Note these concepts and then write a brief summary about how the article incorporates them. In reviews that introduce a study, these can be relatively short. In stand-alone reviews, there may be significantly more texts and more concepts.

Step 5: Demonstrate how these concepts in the literature relate to what you discovered in your study or how the literature connects the concepts or topics being discussed.

In a literature review intro for an article, this information might include a summary of the results or methods of previous studies that correspond to and/or confirm those sections in your own study. For a stand-alone literature review, this may mean highlighting the concepts in each article and showing how they strengthen a hypothesis or show a pattern.

Discuss unaddressed issues in previous studies. These studies that are missing something you address are important to include in your literature review. In addition, those works whose theories and conclusions directly support your findings will be valuable to review here.

Step 6: Identify relationships in the literature and develop and connect your own ideas to them.

This is essentially the same as step 5 but focused on the connections between the literature and the current study or guiding concepts or arguments of the paper, not only on the connections between the works themselves.

Your hypothesis, argument, or guiding concept is the “golden thread” that will ultimately tie the works together and provide readers with specific insights they didn’t have before reading your literature review. Make sure you know where to put the research question , hypothesis, or statement of the problem in your research paper so that you guide your readers logically and naturally from your introduction of earlier work and evidence to the conclusions you want them to draw from the bigger picture.

Your literature review will not only cover publications on your topics but will include your own ideas and contributions. By following these steps you will be telling the specific story that sets the background and shows the significance of your research and you can turn a network of related works into a focused review of the literature.

Literature Review (RRL) Examples

Because creating sample literature reviews would take too long and not properly capture the nuances and detailed information needed for a good review, we have included some links to different types of literature reviews below. You can find links to more literature reviews in these categories by visiting the TUS Library’s website . Sample literature reviews as part of an article, dissertation, or thesis:

  • Critical Thinking and Transferability: A Review of the Literature (Gwendolyn Reece)
  • Building Customer Loyalty: A Customer Experience Based Approach in a Tourism Context (Martina Donnelly)

Sample stand-alone literature reviews

  • Literature Review on Attitudes towards Disability (National Disability Authority)
  • The Effects of Communication Styles on Marital Satisfaction (Hannah Yager)

Additional Literature Review Format Guidelines

In addition to the content guidelines above, authors also need to check which style guidelines to use ( APA , Chicago, MLA, etc.) and what specific rules the target journal might have for how to structure such articles or how many studies to include—such information can usually be found on the journals’ “Guide for Authors” pages. Additionally, use one of the four Wordvice citation generators below, choosing the citation style needed for your paper:

Wordvice Writing and Academic Editing Resources

Finally, after you have finished drafting your literature review, be sure to receive professional proofreading services , including paper editing for your academic work. A competent proofreader who understands academic writing conventions and the specific style guides used by academic journals will ensure that your paper is ready for publication in your target journal.

See our academic resources for further advice on references in your paper , how to write an abstract , how to write a research paper title, how to impress the editor of your target journal with a perfect cover letter , and dozens of other research writing and publication topics.

Logo for BCcampus Open Publishing

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 2: Getting Started in Research

Reviewing the Research Literature

Learning Objectives

  • Define the research literature in psychology and give examples of sources that are part of the research literature and sources that are not.
  • Describe and use several methods for finding previous research on a particular research idea or question.

Reviewing the research literature means finding, reading, and summarizing the published research relevant to your question. An empirical research report written in American Psychological Association (APA) style always includes a written literature review, but it is important to review the literature early in the research process for several reasons.

  • It can help you turn a research idea into an interesting research question.
  • It can tell you if a research question has already been answered.
  • It can help you evaluate the interestingness of a research question.
  • It can give you ideas for how to conduct your own study.
  • It can tell you how your study fits into the research literature.

What Is the Research Literature?

The  research literature  in any field is all the published research in that field. The research literature in psychology is enormous—including millions of scholarly articles and books dating to the beginning of the field—and it continues to grow. Although its boundaries are somewhat fuzzy, the research literature definitely does not include self-help and other pop psychology books, dictionary and encyclopedia entries, websites, and similar sources that are intended mainly for the general public. These are considered unreliable because they are not reviewed by other researchers and are often based on little more than common sense or personal experience. Wikipedia contains much valuable information, but the fact that its authors are anonymous and may not have any formal training or expertise in that subject area, and its content continually changes makes it unsuitable as a basis of sound scientific research. For our purposes, it helps to define the research literature as consisting almost entirely of two types of sources: articles in professional journals, and scholarly books in psychology and related fields.

Professional Journals

Professional journals  are periodicals that publish original research articles. There are thousands of professional journals that publish research in psychology and related fields. They are usually published monthly or quarterly in individual issues, each of which contains several articles. The issues are organized into volumes, which usually consist of all the issues for a calendar year. Some journals are published in hard copy only, others in both hard copy and electronic form, and still others in electronic form only.

Most articles in professional journals are one of two basic types: empirical research reports and review articles.  Empirical research reports  describe one or more new empirical studies conducted by the authors. They introduce a research question, explain why it is interesting, review previous research, describe their method and results, and draw their conclusions. Review articles  summarize previously published research on a topic and usually present new ways to organize or explain the results. When a review article is devoted primarily to presenting a new theory, it is often referred to as a theoretical article .

Figure 2.6 Small Sample of the Thousands of Professional Journals That Publish Research in Psychology and Related Fields

Most professional journals in psychology undergo a process of  double-blind peer review . Researchers who want to publish their work in the journal submit a manuscript to the editor—who is generally an established researcher too—who in turn sends it to two or three experts on the topic. Each reviewer reads the manuscript, writes a critical but constructive review, and sends the review back to the editor along with his or her recommendations. The editor then decides whether to accept the article for publication, ask the authors to make changes and resubmit it for further consideration, or reject it outright. In any case, the editor forwards the reviewers’ written comments to the researchers so that they can revise their manuscript accordingly. This entire process is double-blind, as the reviewers do not know the identity of the researcher(s), and vice versa. Double-blind peer review is helpful because it ensures that the work meets basic standards of the field before it can enter the research literature. However, in order to increase transparency and accountability some newer open access journals (e.g., Frontiers in Psychology) utilize an open peer review process wherein the identities of the reviewers (which remain concealed during the peer review process) are published alongside the journal article.

Scholarly Books

Scholarly books  are books written by researchers and practitioners mainly for use by other researchers and practitioners. A  monograph  is written by a single author or a small group of authors and usually gives a coherent presentation of a topic much like an extended review article.  Edited volumes have an editor or a small group of editors who recruit many authors to write separate chapters on different aspects of the same topic. Although edited volumes can also give a coherent presentation of the topic, it is not unusual for each chapter to take a different perspective or even for the authors of different chapters to openly disagree with each other. In general, scholarly books undergo a peer review process similar to that used by professional journals.

Literature Search Strategies

Using psycinfo and other databases.

The primary method used to search the research literature involves using one or more electronic databases. These include Academic Search Premier, JSTOR, and ProQuest for all academic disciplines, ERIC for education, and PubMed for medicine and related fields. The most important for our purposes, however, is PsycINFO, which is produced by the APA. PsycINFO is so comprehensive—covering thousands of professional journals and scholarly books going back more than 100 years—that for most purposes its content is synonymous with the research literature in psychology. Like most such databases, PsycINFO is usually available through your university library.

PsycINFO consists of individual records for each article, book chapter, or book in the database. Each record includes basic publication information, an abstract or summary of the work (like the one presented at the start of this chapter), and a list of other works cited by that work. A computer interface allows entering one or more search terms and returns any records that contain those search terms. (These interfaces are provided by different vendors and therefore can look somewhat different depending on the library you use.) Each record also contains lists of keywords that describe the content of the work and also a list of index terms. The index terms are especially helpful because they are standardized. Research on differences between women and men, for example, is always indexed under “Human Sex Differences.” Research on notetaking is always indexed under the term “Learning Strategies.” If you do not know the appropriate index terms, PsycINFO includes a thesaurus that can help you find them.

Given that there are nearly four million records in PsycINFO, you may have to try a variety of search terms in different combinations and at different levels of specificity before you find what you are looking for. Imagine, for example, that you are interested in the question of whether women and men differ in terms of their ability to recall experiences from when they were very young. If you were to enter “memory for early experiences” as your search term, PsycINFO would return only six records, most of which are not particularly relevant to your question. However, if you were to enter the search term “memory,” it would return 149,777 records—far too many to look through individually. This is where the thesaurus helps. Entering “memory” into the thesaurus provides several more specific index terms—one of which is “early memories.” While searching for “early memories” among the index terms returns 1,446 records—still too many too look through individually—combining it with “human sex differences” as a second search term returns 37 articles, many of which are highly relevant to the topic.

QR code that links to PsycINFO video

Depending on the vendor that provides the interface to PsycINFO, you may be able to save, print, or e-mail the relevant PsycINFO records. The records might even contain links to full-text copies of the works themselves. (PsycARTICLES is a database that provides full-text access to articles in all journals published by the APA.) If not, and you want a copy of the work, you will have to find out if your library carries the journal or has the book and the hard copy on the library shelves. Be sure to ask a librarian if you need help.

Using Other Search Techniques

QR code that links to Google Scholar video

In addition to entering search terms into PsycINFO and other databases, there are several other techniques you can use to search the research literature. First, if you have one good article or book chapter on your topic—a recent review article is best—you can look through the reference list of that article for other relevant articles, books, and book chapters. In fact, you should do this with any relevant article or book chapter you find. You can also start with a classic article or book chapter on your topic, find its record in PsycINFO (by entering the author’s name or article’s title as a search term), and link from there to a list of other works in PsycINFO that cite that classic article. This works because other researchers working on your topic are likely to be aware of the classic article and cite it in their own work. You can also do a general Internet search using search terms related to your topic or the name of a researcher who conducts research on your topic. This might lead you directly to works that are part of the research literature (e.g., articles in open-access journals or posted on researchers’ own websites). The search engine Google Scholar is especially useful for this purpose. A general Internet search might also lead you to websites that are not part of the research literature but might provide references to works that are. Finally, you can talk to people (e.g., your instructor or other faculty members in psychology) who know something about your topic and can suggest relevant articles and book chapters.

What to Search For

When you do a literature review, you need to be selective. Not every article, book chapter, and book that relates to your research idea or question will be worth obtaining, reading, and integrating into your review. Instead, you want to focus on sources that help you do four basic things: (a) refine your research question, (b) identify appropriate research methods, (c) place your research in the context of previous research, and (d) write an effective research report. Several basic principles can help you find the most useful sources.

First, it is best to focus on recent research, keeping in mind that what counts as recent depends on the topic. For newer topics that are actively being studied, “recent” might mean published in the past year or two. For older topics that are receiving less attention right now, “recent” might mean within the past 10 years. You will get a feel for what counts as recent for your topic when you start your literature search. A good general rule, however, is to start with sources published in the past five years. The main exception to this rule would be classic articles that turn up in the reference list of nearly every other source. If other researchers think that this work is important, even though it is old, then by all means you should include it in your review.

Second, you should look for review articles on your topic because they will provide a useful overview of it—often discussing important definitions, results, theories, trends, and controversies—giving you a good sense of where your own research fits into the literature. You should also look for empirical research reports addressing your question or similar questions, which can give you ideas about how to operationally define your variables and collect your data. As a general rule, it is good to use methods that others have already used successfully unless you have good reasons not to. Finally, you should look for sources that provide information that can help you argue for the interestingness of your research question. For a study on the effects of cell phone use on driving ability, for example, you might look for information about how widespread cell phone use is, how frequent and costly motor vehicle crashes are, and so on.

How many sources are enough for your literature review? This is a difficult question because it depends on how extensively your topic has been studied and also on your own goals. One study found that across a variety of professional journals in psychology, the average number of sources cited per article was about 50 (Adair & Vohra, 2003) [1] . This gives a rough idea of what professional researchers consider to be adequate. As a student, you might be assigned a much lower minimum number of references to use, but the principles for selecting the most useful ones remain the same.

Key Takeaways

  • The research literature in psychology is all the published research in psychology, consisting primarily of articles in professional journals and scholarly books.
  • Early in the research process, it is important to conduct a review of the research literature on your topic to refine your research question, identify appropriate research methods, place your question in the context of other research, and prepare to write an effective research report.
  • There are several strategies for finding previous research on your topic. Among the best is using PsycINFO, a computer database that catalogs millions of articles, books, and book chapters in psychology and related fields.
  • Practice: Use the techniques discussed in this section to find 10 journal articles and book chapters on one of the following research ideas: memory for smells, aggressive driving, the causes of narcissistic personality disorder, the functions of the intraparietal sulcus, or prejudice against the physically handicapped.
  • Watch the following video clip produced by UBCiSchool about how to read an academic paper (without losing your mind):

QR code that links to UBCiSchool video

Video Attributions

  • “ Sample PsycINFO Search on EBSCOhost ” by APA Publishing Training . Standard YouTube Licence.
  • “ Using Google Scholar (CLIP) ” by clipinfolit . CC BY (Attribution)
  • “ How to Read an Academic Paper ” by UBCiSchool . CC BY (Attribution)
  • Adair, J. G., & Vohra, N. (2003). The explosion of knowledge, references, and citations: Psychology’s unique response to a crisis. American Psychologist, 58 , 15–23. ↵

All the published research in a particular field.

Periodicals that publish original research articles.

A type of research article which describes one or more new empirical studies conducted by the authors.

A type of research article that summarizes previously published research on a topic and usually presents new ways to organize or explain the results.

A type of review article primarily devoted to presenting a new theory.

Books written by researchers and practitioners mainly for sue by other researchers and practitioners.

Type of scholarly book written by a single author or small group of authors, coherently presents a topic much like an extended review article.

A type of scholarly book in which an editor or small group of editors recruit many authors to write separate chapters on different aspects of the same topic.

An electronic database covering thousands of professional journals and scholarly books produced by the APA.

Research Methods in Psychology - 2nd Canadian Edition by Paul C. Price, Rajiv Jhangiani, & I-Chant A. Chiang is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

research literature literature

Logo for University of Central Florida Pressbooks

Chapter Four: Theory, Methodologies, Methods, and Evidence

Research Methods

This page discusses the following topics:

Research Goals

Research method types.

Before discussing research   methods , we need to distinguish them from  methodologies  and  research skills . Methodologies, linked to literary theories, are tools and lines of investigation: sets of practices and propositions about texts and the world. Researchers using Marxist literary criticism will adopt methodologies that look to material forces like labor, ownership, and technology to understand literature and its relationship to the world. They will also seek to understand authors not as inspired geniuses but as people whose lives and work are shaped by social forces.

Example: Critical Race Theory Methodologies

Critical Race Theory may use a variety of methodologies, including

  • Interest convergence: investigating whether marginalized groups only achieve progress when dominant groups benefit as well
  • Intersectional theory: investigating how multiple factors of advantage and disadvantage around race, gender, ethnicity, religion, etc. operate together in complex ways
  • Radical critique of the law: investigating how the law has historically been used to marginalize particular groups, such as black people, while recognizing that legal efforts are important to achieve emancipation and civil rights
  • Social constructivism: investigating how race is socially constructed (rather than biologically grounded)
  • Standpoint epistemology: investigating how knowledge relates to social position
  • Structural determinism: investigating how structures of thought and of organizations determine social outcomes

To identify appropriate methodologies, you will need to research your chosen theory and gather what methodologies are associated with it. For the most part, we can’t assume that there are “one size fits all” methodologies.

Research skills are about how you handle materials such as library search engines, citation management programs, special collections materials, and so on.

Research methods  are about where and how you get answers to your research questions. Are you conducting interviews? Visiting archives? Doing close readings? Reviewing scholarship? You will need to choose which methods are most appropriate to use in your research and you need to gain some knowledge about how to use these methods. In other words, you need to do some research into research methods!

Your choice of research method depends on the kind of questions you are asking. For example, if you want to understand how an author progressed through several drafts to arrive at a final manuscript, you may need to do archival research. If you want to understand why a particular literary work became a bestseller, you may need to do audience research. If you want to know why a contemporary author wrote a particular work, you may need to do interviews. Usually literary research involves a combination of methods such as  archival research ,  discourse analysis , and  qualitative research  methods.

Literary research methods tend to differ from research methods in the hard sciences (such as physics and chemistry). Science research must present results that are reproducible, while literary research rarely does (though it must still present evidence for its claims). Literary research often deals with questions of meaning, social conventions, representations of lived experience, and aesthetic effects; these are questions that reward dialogue and different perspectives rather than one great experiment that settles the issue. In literary research, we might get many valuable answers even though they are quite different from one another. Also in literary research, we usually have some room to speculate about answers, but our claims have to be plausible (believable) and our argument comprehensive (meaning we don’t overlook evidence that would alter our argument significantly if it were known).

A literary researcher might select the following:

Theory: Critical Race Theory

Methodology: Social Constructivism

Method: Scholarly

Skills: Search engines, citation management

Wendy Belcher, in  Writing Your Journal Article in 12 Weeks , identifies two main approaches to understanding literary works: looking at a text by itself (associated with New Criticism ) and looking at texts as they connect to society (associated with Cultural Studies ). The goal of New Criticism is to bring the reader further into the text. The goal of Cultural Studies is to bring the reader into the network of discourses that surround and pass through the text. Other approaches, such as Ecocriticism, relate literary texts to the Sciences (as well as to the Humanities).

The New Critics, starting in the 1940s,  focused on meaning within the text itself, using a method they called “ close reading .” The text itself becomes e vidence for a particular reading. Using this approach, you should summarize the literary work briefly and q uote particularly meaningful passages, being sure to introduce quotes and then interpret them (never let them stand alone). Make connections within the work; a sk  “why” and “how” the various parts of the text relate to each other.

Cultural Studies critics see all texts  as connected to society; the critic  therefore has to connect a text to at least one political or social issue. How and why does  the text reproduce particular knowledge systems (known as discourses) and how do these knowledge systems relate to issues of power within the society? Who speaks and when? Answering these questions helps your reader understand the text in context. Cultural contexts can include the treatment of gender (Feminist, Queer), class (Marxist), nationality, race, religion, or any other area of human society.

Other approaches, such as psychoanalytic literary criticism , look at literary texts to better understand human psychology. A psychoanalytic reading can focus on a character, the author, the reader, or on society in general. Ecocriticism  look at human understandings of nature in literary texts.

We select our research methods based on the kinds of things we want to know. For example, we may be studying the relationship between literature and society, between author and text, or the status of a work in the literary canon. We may want to know about a work’s form, genre, or thematics. We may want to know about the audience’s reading and reception, or about methods for teaching literature in schools.

Below are a few research methods and their descriptions. You may need to consult with your instructor about which ones are most appropriate for your project. The first list covers methods most students use in their work. The second list covers methods more commonly used by advanced researchers. Even if you will not be using methods from this second list in your research project, you may read about these research methods in the scholarship you find.

Most commonly used undergraduate research methods:

  • Scholarship Methods:  Studies the body of scholarship written about a particular author, literary work, historical period, literary movement, genre, theme, theory, or method.
  • Textual Analysis Methods:  Used for close readings of literary texts, these methods also rely on literary theory and background information to support the reading.
  • Biographical Methods:  Used to study the life of the author to better understand their work and times, these methods involve reading biographies and autobiographies about the author, and may also include research into private papers, correspondence, and interviews.
  • Discourse Analysis Methods:  Studies language patterns to reveal ideology and social relations of power. This research involves the study of institutions, social groups, and social movements to understand how people in various settings use language to represent the world to themselves and others. Literary works may present complex mixtures of discourses which the characters (and readers) have to navigate.
  • Creative Writing Methods:  A literary re-working of another literary text, creative writing research is used to better understand a literary work by investigating its language, formal structures, composition methods, themes, and so on. For instance, a creative research project may retell a story from a minor character’s perspective to reveal an alternative reading of events. To qualify as research, a creative research project is usually combined with a piece of theoretical writing that explains and justifies the work.

Methods used more often by advanced researchers:

  • Archival Methods: Usually involves trips to special collections where original papers are kept. In these archives are many unpublished materials such as diaries, letters, photographs, ledgers, and so on. These materials can offer us invaluable insight into the life of an author, the development of a literary work, or the society in which the author lived. There are at least three major archives of James Baldwin’s papers: The Smithsonian , Yale , and The New York Public Library . Descriptions of such materials are often available online, but the materials themselves are typically stored in boxes at the archive.
  • Computational Methods:  Used for statistical analysis of texts such as studies of the popularity and meaning of particular words in literature over time.
  • Ethnographic Methods:  Studies groups of people and their interactions with literary works, for instance in educational institutions, in reading groups (such as book clubs), and in fan networks. This approach may involve interviews and visits to places (including online communities) where people interact with literary works. Note: before you begin such work, you must have  Institutional Review Board (IRB)  approval “to protect the rights and welfare of human participants involved in research.”
  • Visual Methods:  Studies the visual qualities of literary works. Some literary works, such as illuminated manuscripts, children’s literature, and graphic novels, present a complex interplay of text and image. Even works without illustrations can be studied for their use of typography, layout, and other visual features.

Regardless of the method(s) you choose, you will need to learn how to apply them to your work and how to carry them out successfully. For example, you should know that many archives do not allow you to bring pens (you can use pencils) and you may not be allowed to bring bags into the archives. You will need to keep a record of which documents you consult and their location (box number, etc.) in the archives. If you are unsure how to use a particular method, please consult a book about it. [1] Also, ask for the advice of trained researchers such as your instructor or a research librarian.

  • What research method(s) will you be using for your paper? Why did you make this method selection over other methods? If you haven’t made a selection yet, which methods are you considering?
  • What specific methodological approaches are you most interested in exploring in relation to the chosen literary work?
  • What is your plan for researching your method(s) and its major approaches?
  • What was the most important lesson you learned from this page? What point was confusing or difficult to understand?

Write your answers in a webcourse discussion page.

research literature literature

  • Introduction to Research Methods: A Practical Guide for Anyone Undertaking a Research Project  by Catherine, Dr. Dawson
  • Practical Research Methods: A User-Friendly Guide to Mastering Research Techniques and Projects  by Catherine Dawson
  • Qualitative Inquiry and Research Design: Choosing Among Five Approaches  by John W. Creswell  Cheryl N. Poth
  • Qualitative Research Evaluation Methods: Integrating Theory and Practice  by Michael Quinn Patton
  • Research Design: Qualitative, Quantitative, and Mixed Methods Approaches  by John W. Creswell  J. David Creswell
  • Research Methodology: A Step-by-Step Guide for Beginners  by Ranjit Kumar
  • Research Methodology: Methods and Techniques  by C.R. Kothari

Strategies for Conducting Literary Research Copyright © 2021 by Barry Mauer & John Venecek is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

  • Privacy Policy
  • SignUp/Login

Research Method

Home » Literature Review – Types Writing Guide and Examples

Literature Review – Types Writing Guide and Examples

Table of Contents

Literature Review

Literature Review

Definition:

A literature review is a comprehensive and critical analysis of the existing literature on a particular topic or research question. It involves identifying, evaluating, and synthesizing relevant literature, including scholarly articles, books, and other sources, to provide a summary and critical assessment of what is known about the topic.

Types of Literature Review

Types of Literature Review are as follows:

  • Narrative literature review : This type of review involves a comprehensive summary and critical analysis of the available literature on a particular topic or research question. It is often used as an introductory section of a research paper.
  • Systematic literature review: This is a rigorous and structured review that follows a pre-defined protocol to identify, evaluate, and synthesize all relevant studies on a specific research question. It is often used in evidence-based practice and systematic reviews.
  • Meta-analysis: This is a quantitative review that uses statistical methods to combine data from multiple studies to derive a summary effect size. It provides a more precise estimate of the overall effect than any individual study.
  • Scoping review: This is a preliminary review that aims to map the existing literature on a broad topic area to identify research gaps and areas for further investigation.
  • Critical literature review : This type of review evaluates the strengths and weaknesses of the existing literature on a particular topic or research question. It aims to provide a critical analysis of the literature and identify areas where further research is needed.
  • Conceptual literature review: This review synthesizes and integrates theories and concepts from multiple sources to provide a new perspective on a particular topic. It aims to provide a theoretical framework for understanding a particular research question.
  • Rapid literature review: This is a quick review that provides a snapshot of the current state of knowledge on a specific research question or topic. It is often used when time and resources are limited.
  • Thematic literature review : This review identifies and analyzes common themes and patterns across a body of literature on a particular topic. It aims to provide a comprehensive overview of the literature and identify key themes and concepts.
  • Realist literature review: This review is often used in social science research and aims to identify how and why certain interventions work in certain contexts. It takes into account the context and complexities of real-world situations.
  • State-of-the-art literature review : This type of review provides an overview of the current state of knowledge in a particular field, highlighting the most recent and relevant research. It is often used in fields where knowledge is rapidly evolving, such as technology or medicine.
  • Integrative literature review: This type of review synthesizes and integrates findings from multiple studies on a particular topic to identify patterns, themes, and gaps in the literature. It aims to provide a comprehensive understanding of the current state of knowledge on a particular topic.
  • Umbrella literature review : This review is used to provide a broad overview of a large and diverse body of literature on a particular topic. It aims to identify common themes and patterns across different areas of research.
  • Historical literature review: This type of review examines the historical development of research on a particular topic or research question. It aims to provide a historical context for understanding the current state of knowledge on a particular topic.
  • Problem-oriented literature review : This review focuses on a specific problem or issue and examines the literature to identify potential solutions or interventions. It aims to provide practical recommendations for addressing a particular problem or issue.
  • Mixed-methods literature review : This type of review combines quantitative and qualitative methods to synthesize and analyze the available literature on a particular topic. It aims to provide a more comprehensive understanding of the research question by combining different types of evidence.

Parts of Literature Review

Parts of a literature review are as follows:

Introduction

The introduction of a literature review typically provides background information on the research topic and why it is important. It outlines the objectives of the review, the research question or hypothesis, and the scope of the review.

Literature Search

This section outlines the search strategy and databases used to identify relevant literature. The search terms used, inclusion and exclusion criteria, and any limitations of the search are described.

Literature Analysis

The literature analysis is the main body of the literature review. This section summarizes and synthesizes the literature that is relevant to the research question or hypothesis. The review should be organized thematically, chronologically, or by methodology, depending on the research objectives.

Critical Evaluation

Critical evaluation involves assessing the quality and validity of the literature. This includes evaluating the reliability and validity of the studies reviewed, the methodology used, and the strength of the evidence.

The conclusion of the literature review should summarize the main findings, identify any gaps in the literature, and suggest areas for future research. It should also reiterate the importance of the research question or hypothesis and the contribution of the literature review to the overall research project.

The references list includes all the sources cited in the literature review, and follows a specific referencing style (e.g., APA, MLA, Harvard).

How to write Literature Review

Here are some steps to follow when writing a literature review:

  • Define your research question or topic : Before starting your literature review, it is essential to define your research question or topic. This will help you identify relevant literature and determine the scope of your review.
  • Conduct a comprehensive search: Use databases and search engines to find relevant literature. Look for peer-reviewed articles, books, and other academic sources that are relevant to your research question or topic.
  • Evaluate the sources: Once you have found potential sources, evaluate them critically to determine their relevance, credibility, and quality. Look for recent publications, reputable authors, and reliable sources of data and evidence.
  • Organize your sources: Group the sources by theme, method, or research question. This will help you identify similarities and differences among the literature, and provide a structure for your literature review.
  • Analyze and synthesize the literature : Analyze each source in depth, identifying the key findings, methodologies, and conclusions. Then, synthesize the information from the sources, identifying patterns and themes in the literature.
  • Write the literature review : Start with an introduction that provides an overview of the topic and the purpose of the literature review. Then, organize the literature according to your chosen structure, and analyze and synthesize the sources. Finally, provide a conclusion that summarizes the key findings of the literature review, identifies gaps in knowledge, and suggests areas for future research.
  • Edit and proofread: Once you have written your literature review, edit and proofread it carefully to ensure that it is well-organized, clear, and concise.

Examples of Literature Review

Here’s an example of how a literature review can be conducted for a thesis on the topic of “ The Impact of Social Media on Teenagers’ Mental Health”:

  • Start by identifying the key terms related to your research topic. In this case, the key terms are “social media,” “teenagers,” and “mental health.”
  • Use academic databases like Google Scholar, JSTOR, or PubMed to search for relevant articles, books, and other publications. Use these keywords in your search to narrow down your results.
  • Evaluate the sources you find to determine if they are relevant to your research question. You may want to consider the publication date, author’s credentials, and the journal or book publisher.
  • Begin reading and taking notes on each source, paying attention to key findings, methodologies used, and any gaps in the research.
  • Organize your findings into themes or categories. For example, you might categorize your sources into those that examine the impact of social media on self-esteem, those that explore the effects of cyberbullying, and those that investigate the relationship between social media use and depression.
  • Synthesize your findings by summarizing the key themes and highlighting any gaps or inconsistencies in the research. Identify areas where further research is needed.
  • Use your literature review to inform your research questions and hypotheses for your thesis.

For example, after conducting a literature review on the impact of social media on teenagers’ mental health, a thesis might look like this:

“Using a mixed-methods approach, this study aims to investigate the relationship between social media use and mental health outcomes in teenagers. Specifically, the study will examine the effects of cyberbullying, social comparison, and excessive social media use on self-esteem, anxiety, and depression. Through an analysis of survey data and qualitative interviews with teenagers, the study will provide insight into the complex relationship between social media use and mental health outcomes, and identify strategies for promoting positive mental health outcomes in young people.”

Reference: Smith, J., Jones, M., & Lee, S. (2019). The effects of social media use on adolescent mental health: A systematic review. Journal of Adolescent Health, 65(2), 154-165. doi:10.1016/j.jadohealth.2019.03.024

Reference Example: Author, A. A., Author, B. B., & Author, C. C. (Year). Title of article. Title of Journal, volume number(issue number), page range. doi:0000000/000000000000 or URL

Applications of Literature Review

some applications of literature review in different fields:

  • Social Sciences: In social sciences, literature reviews are used to identify gaps in existing research, to develop research questions, and to provide a theoretical framework for research. Literature reviews are commonly used in fields such as sociology, psychology, anthropology, and political science.
  • Natural Sciences: In natural sciences, literature reviews are used to summarize and evaluate the current state of knowledge in a particular field or subfield. Literature reviews can help researchers identify areas where more research is needed and provide insights into the latest developments in a particular field. Fields such as biology, chemistry, and physics commonly use literature reviews.
  • Health Sciences: In health sciences, literature reviews are used to evaluate the effectiveness of treatments, identify best practices, and determine areas where more research is needed. Literature reviews are commonly used in fields such as medicine, nursing, and public health.
  • Humanities: In humanities, literature reviews are used to identify gaps in existing knowledge, develop new interpretations of texts or cultural artifacts, and provide a theoretical framework for research. Literature reviews are commonly used in fields such as history, literary studies, and philosophy.

Role of Literature Review in Research

Here are some applications of literature review in research:

  • Identifying Research Gaps : Literature review helps researchers identify gaps in existing research and literature related to their research question. This allows them to develop new research questions and hypotheses to fill those gaps.
  • Developing Theoretical Framework: Literature review helps researchers develop a theoretical framework for their research. By analyzing and synthesizing existing literature, researchers can identify the key concepts, theories, and models that are relevant to their research.
  • Selecting Research Methods : Literature review helps researchers select appropriate research methods and techniques based on previous research. It also helps researchers to identify potential biases or limitations of certain methods and techniques.
  • Data Collection and Analysis: Literature review helps researchers in data collection and analysis by providing a foundation for the development of data collection instruments and methods. It also helps researchers to identify relevant data sources and identify potential data analysis techniques.
  • Communicating Results: Literature review helps researchers to communicate their results effectively by providing a context for their research. It also helps to justify the significance of their findings in relation to existing research and literature.

Purpose of Literature Review

Some of the specific purposes of a literature review are as follows:

  • To provide context: A literature review helps to provide context for your research by situating it within the broader body of literature on the topic.
  • To identify gaps and inconsistencies: A literature review helps to identify areas where further research is needed or where there are inconsistencies in the existing literature.
  • To synthesize information: A literature review helps to synthesize the information from multiple sources and present a coherent and comprehensive picture of the current state of knowledge on the topic.
  • To identify key concepts and theories : A literature review helps to identify key concepts and theories that are relevant to your research question and provide a theoretical framework for your study.
  • To inform research design: A literature review can inform the design of your research study by identifying appropriate research methods, data sources, and research questions.

Characteristics of Literature Review

Some Characteristics of Literature Review are as follows:

  • Identifying gaps in knowledge: A literature review helps to identify gaps in the existing knowledge and research on a specific topic or research question. By analyzing and synthesizing the literature, you can identify areas where further research is needed and where new insights can be gained.
  • Establishing the significance of your research: A literature review helps to establish the significance of your own research by placing it in the context of existing research. By demonstrating the relevance of your research to the existing literature, you can establish its importance and value.
  • Informing research design and methodology : A literature review helps to inform research design and methodology by identifying the most appropriate research methods, techniques, and instruments. By reviewing the literature, you can identify the strengths and limitations of different research methods and techniques, and select the most appropriate ones for your own research.
  • Supporting arguments and claims: A literature review provides evidence to support arguments and claims made in academic writing. By citing and analyzing the literature, you can provide a solid foundation for your own arguments and claims.
  • I dentifying potential collaborators and mentors: A literature review can help identify potential collaborators and mentors by identifying researchers and practitioners who are working on related topics or using similar methods. By building relationships with these individuals, you can gain valuable insights and support for your own research and practice.
  • Keeping up-to-date with the latest research : A literature review helps to keep you up-to-date with the latest research on a specific topic or research question. By regularly reviewing the literature, you can stay informed about the latest findings and developments in your field.

Advantages of Literature Review

There are several advantages to conducting a literature review as part of a research project, including:

  • Establishing the significance of the research : A literature review helps to establish the significance of the research by demonstrating the gap or problem in the existing literature that the study aims to address.
  • Identifying key concepts and theories: A literature review can help to identify key concepts and theories that are relevant to the research question, and provide a theoretical framework for the study.
  • Supporting the research methodology : A literature review can inform the research methodology by identifying appropriate research methods, data sources, and research questions.
  • Providing a comprehensive overview of the literature : A literature review provides a comprehensive overview of the current state of knowledge on a topic, allowing the researcher to identify key themes, debates, and areas of agreement or disagreement.
  • Identifying potential research questions: A literature review can help to identify potential research questions and areas for further investigation.
  • Avoiding duplication of research: A literature review can help to avoid duplication of research by identifying what has already been done on a topic, and what remains to be done.
  • Enhancing the credibility of the research : A literature review helps to enhance the credibility of the research by demonstrating the researcher’s knowledge of the existing literature and their ability to situate their research within a broader context.

Limitations of Literature Review

Limitations of Literature Review are as follows:

  • Limited scope : Literature reviews can only cover the existing literature on a particular topic, which may be limited in scope or depth.
  • Publication bias : Literature reviews may be influenced by publication bias, which occurs when researchers are more likely to publish positive results than negative ones. This can lead to an incomplete or biased picture of the literature.
  • Quality of sources : The quality of the literature reviewed can vary widely, and not all sources may be reliable or valid.
  • Time-limited: Literature reviews can become quickly outdated as new research is published, making it difficult to keep up with the latest developments in a field.
  • Subjective interpretation : Literature reviews can be subjective, and the interpretation of the findings can vary depending on the researcher’s perspective or bias.
  • Lack of original data : Literature reviews do not generate new data, but rather rely on the analysis of existing studies.
  • Risk of plagiarism: It is important to ensure that literature reviews do not inadvertently contain plagiarism, which can occur when researchers use the work of others without proper attribution.

About the author

' src=

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Research Paper Conclusion

Research Paper Conclusion – Writing Guide and...

Appendices

Appendices – Writing Guide, Types and Examples

Research Report

Research Report – Example, Writing Guide and...

Delimitations

Delimitations in Research – Types, Examples and...

Scope of the Research

Scope of the Research – Writing Guide and...

Research Contribution

Research Contribution – Thesis Guide

Duke University Libraries

Literature Reviews

  • 1. Define your research question
  • Getting started
  • Types of reviews

Define topic

Brainstorming, limit scope, how ai can help, helpful resources.

  • 2. Plan your search
  • 3. Search the literature
  • 4. Organize your results
  • 5. Synthesize your findings
  • 6. Write the review
  • Thompson Writing Studio This link opens in a new window
  • Need to write a systematic review? This link opens in a new window

research literature literature

Contact a Librarian

Ask a Librarian

Mission animated icons created by Freepik - Flaticon

For example, the following research question is too broad.

Does social media affect mental health in college students?

A more focused research question could be:

Does social media play a role in the number of eating disorder diagnoses in college-aged women?

Need some guidance? Contact your subject librarian for a research consultation or reach out to [email protected] .

Start the brainstorming process:

  • Begin by identifying a broad topic of research interest (the more you're personally interested in it, the better!). Write it down to clarify your focus.
  • Narrow down your scope  by considering specific areas within this topic that you want to explore.
  • Identify key themes and elements  related to these specific areas. These will be the building blocks of your investigation.
  • Finally,  integrate these key areas and elements  into a well-crafted research question.

One way to brainstorm is to use a flowchart.

research literature literature

The scope of a topic  encompasses the boundaries and constraints you set for your research. The more precise your research question, the smoother the research process becomes.

Feeling stuck? Here are some ways you can narrow down your topic:

  • geographic location
  • time period
  • discipline/field of study
  • age group, etc.

research literature literature

You:  I need to conduct a literature review about social media, mental health, and college students. What are some example research questions within this topic?

ChatGPT: 

  • How does the frequency and nature of social media use impact the mental health of college students?
  • What role do specific social media platforms play in influencing the mental well-being of college students?
  • How do online social interactions on platforms like Facebook, Instagram, or Twitter relate to the mental health outcomes of college students?
  • Are there differences in the association between social media use and mental health outcomes based on demographic factors such as gender, age, or academic major among college students?
  • What strategies do college students employ to cope with negative mental health effects associated with social media use?
  • How does the perception of social support through online platforms affect the mental health of college students?
  • To what extent does the comparison of oneself to others on social media contribute to mental health challenges among college students?
  • What interventions or educational programs are effective in promoting positive mental health behaviors in the context of social media use for college students?
  • How do social media literacy and awareness programs impact the mental health attitudes and behaviors of college students?
  • What is the relationship between the use of social media for academic and social purposes and the mental health outcomes of college students?

Generative AI tools : ChatGPT , Google Gemini (formerly Bard) , Claude , Microsoft Copilot

For more information on how to incorporate AI tools into your research, check out the section on AI Tools .

  • Developing Your Research Question
  • Developing a Researchable Question
  • << Previous: Types of reviews
  • Next: 2. Plan your search >>
  • Last Updated: Feb 15, 2024 1:45 PM
  • URL: https://guides.library.duke.edu/lit-reviews

Duke University Libraries

Services for...

  • Faculty & Instructors
  • Graduate Students
  • Undergraduate Students
  • International Students
  • Patrons with Disabilities

Twitter

  • Harmful Language Statement
  • Re-use & Attribution / Privacy
  • Support the Libraries

Creative Commons License

research literature literature

Something went wrong when searching for seed articles. Please try again soon.

No articles were found for that search term.

Author, year The title of the article goes here

LITERATURE REVIEW SOFTWARE FOR BETTER RESEARCH

research literature literature

“This tool really helped me to create good bibtex references for my research papers”

Ali Mohammed-Djafari

Director of Research at LSS-CNRS, France

“Any researcher could use it! The paper recommendations are great for anyone and everyone”

Swansea University, Wales

“As a student just venturing into the world of lit reviews, this is a tool that is outstanding and helping me find deeper results for my work.”

Franklin Jeffers

South Oregon University, USA

“One of the 3 most promising tools that (1) do not solely rely on keywords, (2) does nice visualizations, (3) is easy to use”

Singapore Management University

“Incredibly useful tool to get to know more literature, and to gain insight in existing research”

KU Leuven, Belgium

“Seeing my literature list as a network enhances my thinking process!”

Katholieke Universiteit Leuven, Belgium

“I can’t live without you anymore! I also recommend you to my students.”

Professor at The Chinese University of Hong Kong

“This has helped me so much in researching the literature. Currently, I am beginning to investigate new fields and this has helped me hugely”

Aran Warren

Canterbury University, NZ

“It's nice to get a quick overview of related literature. Really easy to use, and it helps getting on top of the often complicated structures of referencing”

Christoph Ludwig

Technische Universität Dresden, Germany

“Litmaps is extremely helpful with my research. It helps me organize each one of my projects and see how they relate to each other, as well as to keep up to date on publications done in my field”

Daniel Fuller

Clarkson University, USA

“Litmaps is a game changer for finding novel literature... it has been invaluable for my productivity.... I also got my PhD student to use it and they also found it invaluable, finding several gaps they missed”

Varun Venkatesh

Austin Health, Australia

research literature literature

Mastering Literature Reviews with Litmaps

research literature literature

REVIEW article

A literature review of the research on students’ evaluation of teaching in higher education.

Luying Zhao&#x;

  • 1 Marxism School, China Pharmaceutical University, Nanjing, China
  • 2 School of Health Economics and Management, Nanjing University of Chinese Medicine, Nanjing, China
  • 3 School of Science, China Pharmaceutical University, Nanjing, China

Students’ evaluation of teaching is a teaching quality evaluation method and teacher performance evaluation tool commonly used in Chinese and foreign universities, and it is also a controversial hot issue in the field of teaching evaluation. At present, the research results of students’ evaluation of teaching in higher education are relatively rich, mainly focusing on reliability, validity and its influencing factors, construction of index system, problems in practical application and improvement strategies. The purpose of this article is to study the relevant research results of the current Chinese and foreign academic circles, in order to provide useful inspiration for the construction of the index system and practical application of the ideological and political theory course evaluation and teaching of Chinese college students.

Introduction

Students’ evaluations of teaching (SET) is an activity for students to evaluate teachers’ teaching effect and teaching quality, including the reliability, validity, content, form, organization, and management of teaching evaluation. In the 1920s, the earliest college student evaluation system in the world began in the United States. In 1915, Purdue University in the United States gave birth to the first student evaluation scale, and in 1927 began to use the standardized student evaluation scale to evaluate teachers’ teaching, which is considered to be the beginning of the student evaluation system ( De Neve, 1991 ; Theall et al., 2001 ). After the 1980s, the college student evaluation system began to be introduced into China while it was widely used in famous universities in western countries and became an important part of the western education system ( Tu et al., 2019 ). In 2001, the Ministry of education of the people’s Republic of China issued several opinions on Strengthening Undergraduate Teaching in Colleges and universities and improving teaching quality, which clearly pointed out that students should be involved in teaching management. Many colleges and universities across the country responded positively and gradually applied student evaluation to teaching management. Relevant research was gradually enriched, and many suggestions on student evaluation were gradually adopted and implemented by colleges and universities ( Wei and Liu, 2013 ). This article will systematically analyze the relevant theoretical achievements of the current Chinese and foreign academic circles, especially the European and American World College Students’ evaluation of teaching, in order to provide useful enlightenment for the construction of the index system and practical application of the evaluation of Ideological and political theory courses for Chinese college students.

Research methodology: Literature analysis and logical analysis

This article is a literature review, so two main approaches have been adopted: documentary analysis and logical analysis. In terms of literature analysis, a large amount of literature has been consulted in writing this article, and as there is a large body of literature relating to Students’ evaluation of teaching in higher education, the authors has followed three principles in selecting literature to read. The first is to look at the time of publication of the literature, with priority given to those published recently; the second is to look at journals and authors, with priority given to well-known journals and authors; the third is to look at citation rates, with priority given to those with high citation rates; and the fourth is to pay particular attention to the two types of articles that hold pro and con views on SET. In terms of logical analysis, this article argues that the three most critical factors associated with SET are the reliability and validity of SET, the indicator system of SET, the problems that arise in the application of SET, and the countermeasures taken. This article argues that, as a review, the four most critical factors related to SET are reliability and validity, indicator systems, problems arising in application and countermeasures to be taken, and evaluation of the above perspectives. Therefore, the logical framework of this article is: an analysis of the reliability and validity of relevant SETs in the existing literature, an analysis of the indicator system, and an analysis of the problems associated with their practical application, an evaluation of the above-mentioned views on relevant SETs, and finally, a conclusion and recommendations.

Research on the reliability and validity of students’ evaluation of teaching in higher education

In students’ evaluation of teaching, reliability refers to the degree to which students’ evaluation of teaching can stably reflect teachers’ actual teaching level, which is manifested in the stability or consistency of the evaluation results; Validity refers to whether students’ evaluation of teaching can achieve the expected goals and effects ( Hong, 2010 ). Whether, it is reliable and effective is directly related to whether students’ teaching evaluation can be applied to teachers’ teaching evaluation. According to the current research results of the academic circles, although there are many doubts about the reliability of College Students’ teaching evaluation, the traditional view that its reliability is high has not been overturned. The validity is also controversial ( Uttl, 2021 ). The mainstream view is that it is effective on the whole and has been supported by abundant literature.

Views on the reliability of student’s evaluation of teaching in higher education

A skeptical view of the reliability of student’s evaluation of teaching.

In the early research, due to the limitation of research design and method, scholars mostly used the average score of the class to measure the reliability of students’ teaching evaluation. Since this method ignored the differences between individual students, it exaggerated the students’ evaluation to a certain extent. Religious reliability ( Hocevar, 1991 ). With the development and application of statistics and data analysis methods, scholars began to use more scientific measurement tools to conduct empirical research on students’ teaching evaluation reliability. Cheng and Zhang (2016) tested the reliability of the samples from three levels: “inter-student reliability,” “intra-course consistency” and “inter-item reliability,” and concluded that the reliability index inflated due to scoring inertia It cannot explain the reliability of the teaching evaluation results, but shows that the reliability measurement contains more interference information. Gao et al. (2010) and other scholars used the intraclass correlation coefficient to comprehensively evaluate the reliability of students’ teaching evaluation and found that: in various indicators, students’ scores on teachers’ teaching are relatively consistent, so they can be It is judged that its rater reliability is high.

A favorable view of the reliability of student’s evaluation of teaching in higher education

In contrast to Gao et al. (2010) and Morley (2012) and other scholars used the intra class correlation coefficient to comprehensively evaluate the reliability of students’ teaching evaluation and found that students’ scores of teachers’ teaching are relatively consistent in all indicators, so it can be judged that their raters have high reliability also believe that the SET tool used in universities is reasonable, reliable, and effective.

Viewpoints on the validity of student’s evaluation of teaching in higher education

View of sufficient effectiveness.

He (2017) pointed out in his research that college students, as classroom participants and stakeholders, have the most say in the teaching effect, and have the necessary cognitive and judgment skills, so students’ evaluation of teaching is scientific, objective and accurate. Li et al. (2017) believe that compared with other teaching evaluation models, student evaluation of teaching has a more direct and economical advantage, and establishes a teaching system that is mainly based on student evaluation and supplemented by expert evaluation and peer evaluation. A quality assurance system is available. Foreign studies have also pointed out that, from the long-term practice of the student evaluation system in many colleges and universities, although there are doubts, its effectiveness is worthy of recognition ( Chau, 1997 ). Numerous colleges and universities in North America, Europe, and Asia are using student evaluations as a valid indicator to measure teaching effectiveness, or as one of the determinants of teacher promotion, tenure, pay-for-performance, or professional development ( Chen and Hoshower, 2003 ). Cashin and Downey (1992) even argue that student evaluations are more reliable and valid than any other data and can be used to improve teachers’ teaching.

View of insufficient effectiveness

Wang and Guan (2017) and Zhou and Qin (2018) believe that students’ teaching evaluation is students’ subjective value judgment of teachers’ teaching. Students may have unclear cognition of teaching evaluation or negative random evaluation, which leads to the deviation of teaching evaluation results and is difficult to truly reflect the problems in teaching practice. Gu et al. (2021) believes that students are still in the process of knowledge accumulation, and the dislocation of teaching evaluation subjects and perspectives caused by students’ teaching evaluation makes it difficult for them to accurately grasp the information of teaching activities, resulting in evaluation distortion. Morley (2012) , Spooren et al. (2013) and other foreign scholars pointed out that although the methods of measuring the effectiveness of students’ teaching evaluation in some typical studies are widely spread, some of them have logical problems, and educators have only reached a consensus on some characteristics of proving the effectiveness of teaching. Based on these characteristics, the effectiveness of students’ teaching evaluation cannot be clearly defined. Galbraith et al. (2012) also believes that the existing evidence is insufficient to support the effectiveness of SET as a general indicator to evaluate the teaching effect or student learning effect. This paragraph should be deleted) Wolfgang Stroebe (2020) also thinks that the existing evidence shows that students’ evaluation of teaching (sets) can not measure the teaching effect.

Factors affecting the reliability and validity of student’s evaluation of teaching in higher education

Since reliability is a necessary condition for validity, the effectiveness of student evaluation of teaching needs to be supported by reliable evaluation results, so the academic circles generally consider the factors affecting reliability and validity comprehensively. According to the current research results, the influencing factors can be divided into two categories: teaching factors and non-teaching factors. Teaching factors include teaching methods, teaching contents, teaching attitudes, teaching means, etc. Non-teaching factors include students’ individual factors, such as grade, gender, specialty, academic achievement, teaching evaluation attitude, etc.; teachers’ personal factors, such as teachers’ age, gender, professional title, teachers’ favorite degree by students ( Dennis, 2022 ); and curriculum factors, such as course form, course time, course importance, course difficulty, etc. As the teaching factors themselves belong to the content covered by the students’ evaluation of teaching, their influence on the evaluation results is positive. Therefore, the discussion of the influencing factors in the academic circles mainly focuses on the non-teaching factors that cause the deviation of the evaluation results.

Chinese scholars’ research on the influencing factors of College Students’ teaching evaluation is mainly to collect the data of influencing factor assumptions from students through questionnaires and interviews, and combined with the teaching evaluation results of specific colleges and universities, use statistical methods to select appropriate models for data analysis, so as to draw conclusions. Pan and Zhang (2016) concluded through empirical research that students’ subjective cognitive factors have a greater impact on the effectiveness of teaching evaluation than objective factors such as grade, gender, academic achievement, and so on. Li and Meng (2020) used the research method of grounded theory to draw a conclusion that students’ evaluation of teaching is affected by four factors: students, teachers, schools, and courses. If it is not handled properly, it is prone to adverse selection, which affects the effectiveness of teaching evaluation and the quality of school teaching. Long (2019) pointed out after analyzing the teaching evaluation data of students in Shantou University business school that there is no inevitable positive correlation between the teaching evaluation scores obtained by teachers and students’ grades of the course, and the teaching workload of teachers has a significant negative impact on the teaching evaluation scores.

Western scholars’ research on the influencing factors of students’ teaching evaluation is more comprehensive, systematic, and in-depth than domestic. However, because the influencing factors of students’ teaching evaluation are too numerous, and there are certain differences in the survey objects selected by different students, there is no agreement on the degree of influence of each factor. Gallagher (2000) , Ginexi (2003) , Heckert et al. (2006) , and other scholars found through research that students’ characteristics (such as gender, personality, expected score of curriculum, emotion toward teachers, grade, learning expectation, major, attitude toward curriculum and teaching evaluation, the proportion of students participating in teaching evaluation in the total number of students, confidence in the effectiveness and influence of their teaching evaluation results, etc.), Teachers’ characteristics (such as gender, age, educational background, rank, relationship with students, charm and image, etc.) and curriculum characteristics (such as curriculum time, class size, curriculum nature, assessment form, etc.), and even whether the evaluation of teaching is anonymous, and whether the evaluated teachers participate in the evaluation process may have varying degrees of impact on the evaluation behavior of college students in a specific way ( Kekale, 2000 ).

Research on the indicators of student’s evaluation of teaching in higher education

In the process of college students’ teaching evaluation, reasonable teaching evaluation indicators are particularly important. It plays a key role in the accuracy and influence of teaching evaluation results, and is the premise and basis for students’ teaching evaluation to help improve teaching quality. The research and analysis of college students’ teaching evaluation index includes not only the construction of the specific content of the teaching evaluation index, but also the discussion of the theoretical principles that should be followed in the construction of the index. As the central link of college students’ teaching evaluation, the research on teaching evaluation indicators will provide valuable reference for improving students’ teaching evaluation system.

Principles of constructing teaching evaluation indicators for college students

According to the current research results, the most common view in the academic circle on the construction principle of teaching evaluation indicators is to adhere to the “student-centered.” The view of “student-centered” originated from the “child-centered theory” of American educator and psychologist John Dewey, which emphasizes that the essential purpose of education is to promote the comprehensive and harmonious development of students ( Ye, 2000 ). The student-centered evaluation index system requires that the evaluation scale should be designed from the perspective of students, based on students’ cognitive level and actual needs, and based on students’ real feelings and gains. Students’ development should become the starting point and foothold of building the teaching evaluation index system ( Lv, 2014 ).

In addition to the mainstream views, scholars such as Wu et al. (2015) also believe that the design of the index system of college students’ teaching evaluation should at least include the characteristics of orientation, academic, interaction, difference, measurability and growth. Jiang and Xiong (2021) pointed out that after analyzing the evaluation indicators of four national universities in Japan, students’ learning behavior and emotional investment should be included in the evaluation index system, and more students’ “learning” should be included in the evaluation field. Tsou (2020) proposed to use AHP to integrate student evaluation, expert evaluation, and regular teaching assessment into the teaching evaluation system to form a new method of “same platform evaluation.” Ching (2019) believes that in order to develop relevant and constructive set indicators, the participation of important stakeholders, such as school managers, teachers and students, is essential, and more importantly, the service attributes that students want (power, rich experience and experience) should be taken into account.

Contents of the teaching evaluation indicators of college students

In the specific content design of college students’ teaching evaluation indicators, Chinese scholars generally agree with the setting mode of secondary indicators. Yan and Wei (2016) believes that the setting of student evaluation indicators should follow the teaching principles of constructivism theory, highlight the core concept of teacher led and student-centered, design secondary indicators covering six aspects: teaching methods, teaching content, teaching attitude, teacher ethics and style, learning elements, learning effects, and set open questions for students to express their opinions and suggestions. Zhang et al. (2017) started with five first-level indicators of teaching attitude, teaching implementation, Teaching means and methods, teaching ability and level, and teaching effect based on literature research and teaching evaluation experience in colleges and universities. There are 33 secondary evaluation indicators based on learning theory and closely related to teaching quality, covering all aspects of the teaching process. Wu et al. (2015) , from the perspective of systems theory, combined teaching evaluation theory, Chinese and other teaching evaluation cases and empirical research results, and designed a comprehensive index covering teaching enthusiasm, teaching organization, learning value, and teacher-student relationship, teaching content, teaching interaction, homework and assessment of 7 single indicators of evaluation index system. It is also worth mentioning that Zhang et al. (2019) optimized the student evaluation index system based on the new era’s requirements for higher education teaching quality, and constructed an index system of three levels: general education indicators, subject sharing indicators and school specific indicators, and added the relevant contents of “moral education” and “ideological politics” to the general education indicators.

Universities in some countries such as the United Kingdom, the United States, and Australia have set up special teaching evaluation and development institutions, whose members are composed of experts from different disciplines, and experts collectively discuss and formulate standardized student teaching evaluation scales. The evaluation indicators of the scale mainly refer to D. L stufflebeam’s CIPP evaluation model ( Zhou, 2012 ). The CIPP model advocates helping managers (the makers of the indicator system) to systematically obtain and use evaluation feedback information in order to meet their needs or to utilize information resources as much as possible. Most of the colleges and universities in Western countries refer to this model, starting from the traditional student evaluation index, and divide it into three dimensions: background condition, process, and result to design evaluation index to systematically evaluate teacher teaching ( Kellaghan and Stufflebeam, 2003 ; Zhao, 2010 ). Another SEEQ model has also been praised by foreign universities. The SEEQ evaluation index is composed of four parts: core index, characteristics of students and courses, additional index (supplementary questions) and open evaluation. Among them, the core index requires students to evaluate nine parts of teachers’ teaching. These nine parts include Academic, emotional, organizational, collaboration, personal communication, curriculum development, assessment, homework, and overall impression of teachers ( Richardson, 2005 ; Marsh, 2007 ; Schellhase, 2010 ). Chinese scholars Jiang and Lu (2019) also found in their research on the students’ teaching evaluation system in ten first-class foreign universities that Stanford, MIT, Cornell and other colleges and universities evaluate students’ overall experience of the course and achieve the learning goals of the course. The evaluation of the situation and the evaluation of knowledge acquisition and skill development are included in the student evaluation index.

Research on the problems and countermeasures in the practical application of student’s evaluation of teaching in higher education

Since the student evaluation system has been widely used in major colleges and universities in the world, it has not only achieved certain results, but also exposed many problems in practical application. The academic circles have abundant research results on the problems and improvement strategies in the practice of college students’ teaching evaluation. Although the opinions of various scholars are occasionally lacking, they are generally similar. This article summarizes the main points of view.

Problems existing in the practice of teaching evaluation by college students

First, the function of teaching evaluation is alienated. There is a game between teaching managers, teachers and students in the existing student teaching evaluation system, that is, managers focus more on teachers’ “teaching” rather than students’ “learning.” Exaggerating the degree of teaching evaluation’s response to teachers’ teaching level weakens its function of teaching improvement ( Becker and William, 2000 ; Jiang et al., 2018 ; Liang et al., 2020 ). Second, the evaluation index system is unscientific. According to the existing research, the unscientific aspects of teaching evaluation indicators are mainly reflected in the neglect of the subject status of students in teaching evaluation, the failure to distinguish the evaluation indicators of different professional courses, the too many invalid indicators and the complicated content, and the lack of theoretical guidance for the construction of the indicator system, etc ( Ching, 2019 ; Sun, 2021 ). Constantinou and Wijnen-Meijer (2022) also pointed out that students, peers, curriculum managers and self-evaluation should be included in teaching evaluation ( Chan, 2019 ). Third, the use of teaching evaluation results is unreasonable. In many colleges and universities, students’ teaching evaluation is a mere formality, only using quantitative scores to evaluate teachers’ performance, ignoring the value of qualitative teaching evaluation data; at the same time, the processing of teaching evaluation data is too simplistic, and a reasonable result feedback mechanism has not been formed ( Chan, 2019 ). Fourth, the management system is imperfect. Restricted by subjective and objective conditions, at present, the management of students’ teaching evaluation in Colleges and universities at home and abroad is relatively extensive ( Li et al., 2019 ), most of which are implemented by educational administration departments or entrusted to third-party evaluation institutions for operation, and few of them set up special organizations or establish clear rules and regulations to standardize the implementation of teaching evaluation.

Improvement strategies for student’s evaluation of teaching in higher education

In view of the problems existing in the actual operation of College Students’ teaching evaluation, scholars at home and abroad have given suggestions for improvement from different angles. Sun and Sun (2020) believe that the failure of students’ teaching evaluation is caused by various games in teaching evaluation, and the fundamental solution is to change the function from the role of personnel management and summative evaluation of teachers. Tools, transforming into links and means of the ongoing process of diagnosing and developing teacher teaching. Long and Wang (2019) pointed out in their research that the use of the student teaching evaluation system should clarify the value, clarify the standards, and set the rules, and conduct a comprehensive evaluation from the aspects of clarifying the purpose of evaluation, optimizing the evaluation indicators, enriching the evaluation forms, and rationally using the results. It is comprehensively constructed to realize the teaching academic value of students’ evaluation of teaching. Xu (2017) believes that timely self-improvement is an important part of the new student teaching evaluation system, so the student teaching evaluation process should be optimized based on the principle of continuous improvement, and a problem tracking and monitoring guarantee mechanism should be established. Through his research, Svinicki (2010) showed that open evaluation plays an important role in students’ teaching evaluation. The limitation of pure quantitative evaluation on students’ expression should be reduced as much as possible and more open possibilities should be provided in terms of teaching evaluation indicators. Marsh and Herbert (1987) put the perspective on the feedback of teaching evaluation results, and believed that the influence and effectiveness of students’ teaching evaluation results can be expanded through three feedback methods: summary of students’ teaching evaluation results, summary materials for each teacher, and teaching expert advice given in combination with students’ teaching evaluation results. Hassanein et al. (2012) concluded from a SET study conducted in nursing schools that improving the teaching evaluation process must take into account the diversity of student characteristics, student evaluation goals, teaching methods, and institutional context.

Comments on existing research

Research perspective.

The original intention of the student evaluation system is to let students, as the main body of teaching, evaluate teachers’ teaching behavior. However, with the popularization and development of the system in colleges and universities around the world, the conflicts of interest among teaching managers, teachers and students in teaching evaluation are gradually revealed. In this game, managers put the focus of students’ teaching evaluation on teachers, and take teaching evaluation as a simple and effective tool to measure teachers’ performance. In fact, students’ expression of teachers’ teaching is limited and controllable. In the current research results, scholars at home and abroad have a more profound understanding of the absence of students’ evaluation of teaching, generally shifting the research perspective to the concept of “student-centered” evaluation of teaching, and considering “the actual needs of students” and “promoting the all-round development of students” in the research of various parts such as the function, content and results of evaluation of teaching. However, the current research perspective is lack of comprehensiveness, and the seemingly reasonable transformation cannot resolve the contradiction between the three in the student evaluation system. Overemphasizing the student standard will magnify the deviation of the system in the evaluation of teachers’ teaching quality, and increase the cost and burden of teaching managers.

Research contents

The academic research on college students’ teaching evaluation mainly focuses on reliability, validity and its influencing factors, evaluation indicators, deficiencies in practical application and improvement strategies, among which the research results on the effectiveness of teaching evaluation are the most abundant. With the wide application of the student teaching evaluation system, the research scope of validity has expanded from the initial analysis of rationality to the research on the reliability of students’ teaching evaluation results and the final validity of students’ teaching evaluation. In terms of influencing factors, although researchers have formed a relatively unified view on its main categories, due to the large number of subtle factors and different research perspectives, the influence results of specific factors are also different, making the research results of this part complex and full of controversy. In terms of evaluation indicators, the research on the theory of index construction has been relatively complete, and the combination of qualitative and quantitative evaluation can generally be used, which reflects the academic quality of index construction. However, there are still different strengths and weaknesses in the design of specific evaluation indicators, and there is a lack of a comprehensive, systematic and authoritative index design framework, so it is difficult to form a unified opinion and promote its application. In terms of problems and countermeasures in practice, scholars at home and abroad have relatively unanimous opinions on the problems existing in the current teaching evaluation process of college students, and have carried out a relatively comprehensive analysis. However, the countermeasures proposed for the problem are too vague and unconvincing, and it is necessary to further verify and concretize them in the application process to obtain more effective improvement suggestions.

Research methods

At present, scholars’ research on students’ teaching evaluation system is no longer limited to literature, but more to empirical investigation and statistical analysis. In recent years, in terms of the reliability and validity of students’ teaching evaluation and its influencing factors, more and more researchers have used statistical methods such as independence test, multiple regression analysis, ordered regression model (OLM) to analyze the data of students’ teaching evaluation. The correct and reasonable use of appropriate data processing methods has significantly improved the scientificity of the research on the effectiveness of students’ teaching evaluation. In order to build a scientific and reasonable evaluation index, researchers prefer to use questionnaires, interviews, random sampling and other methods to conduct empirical research on the subjects and cases of teaching evaluation in colleges and universities. However, a good empirical study is extremely difficult to operate, which requires a large sample size and will also cost more time, human resources and other resources. Therefore, the evidence of empirical research in the current results is still relatively shallow, and we can try to combine it with big data and artificial intelligence algorithms to supplement it with diversified research methods.

Research trends

According to the current research trend in academia, first of all, researchers will continue to explore the influencing factors of students’ teaching evaluation. The influencing factors of college students’ teaching evaluation are extremely complex, but it is extremely important to overcome the negative effects of interfering factors and improve the limitation of students’ teaching evaluation. Therefore, the research on this issue in academic circles will continue to deepen. Secondly, the rapid development of the Internet has innovated the form of students’ teaching evaluation, and students’ online teaching evaluation has become the current mainstream model. While online teaching evaluation brings convenience to the students’ teaching evaluation system, new problems such as the weakening of the realism of the scene and the difficulty of supervising the process have also appeared. In addition, how to solve the shortcomings of the online teaching evaluation system, such as strong subjectivity of students, low teacher participation, and imperfect application of result feedback, is also becoming a problem worth exploring for researchers. Finally, the integrated development of multi-disciplinary and multi-angle will be the key direction of future research on the teaching evaluation system of college students. Scholars have found in their research that the content involved in the student evaluation system is far beyond the field of education, and its scope also covers psychology, sociology, statistics, economics and many other disciplines. Therefore, some researchers have integrated and analyzed student evaluation of teaching with other disciplines. It is foreseeable that in future research, scholars will view the improvement and development of the student evaluation system from a multidisciplinary perspective.

Conclusion and recommendations

In this article, we have provided a more in-depth analysis of the theories and methods of student’s evaluation of teaching in higher education in China and abroad, and point out the desirable experiences and shortcomings of them. Firstly, in terms of the reliability and validity of student’s evaluation of teaching, the current research has made great progress in terms of validation methods, and some scholars have been able to use appropriate data analysis models to improve the persuasiveness of the results. Secondly, the research on the theory and structure of the evaluation indicators for university students generally revolves around the “student-centered theory” and “secondary indicator structure,” the rationality of which has been confirmed; however, the academic circle has not yet formed a unified opinion or standard on the selection of the specific content of the evaluation indicators. However, there is no unified opinion or standard on the selection of the specific content of evaluation indicators, and fewer scholars have paid attention to the differentiation of the indicators for different courses. Finally, regarding the problems and countermeasures in the application of student’s evaluation of teaching in higher education, current research has analyzed the process of student’s evaluation of teaching from the aspects of purpose, indicator system, application of results and process management in an all-round way and found the shortcomings, however, the scholars’ expressions of improvement measures are still abstract and lack of pertinence, making it difficult to carry out concrete operations. In view of the above analysis, this article improves the research on student’s evaluation in Western higher education from a Chinese perspective, and at the same time provides a reference for the construction of an indicator system for student evaluation in Chinese universities, taking into account the actual situation of Chinese universities. The future research will be based on the successful experience of student evaluation in universities, improve the problems, explore the influence mechanism of different factors on student evaluation in universities, optimize the indicator system and management system of student evaluation, especially the indicator system of student evaluation in ideological and political theory courses in Chinese higher education will be constructed in accordance with the characteristics of Chinese ideological and political theory courses, so as to improve the teaching quality of ideological and political theory courses in Chinese higher education. We will make our contribution to improving the teaching quality of ideological and political theory courses in Chinese higher education.

Author contributions

SY: conceptualization, resources, writing-review and editing, and supervision. SY, LZ, and PX: methodology. LZ and PX: formal analysis. LZ and YC: investigation. SY and YC: project administration. All authors contributed to the article and approved the submitted version.

This research was funded by National Innovation and Entrepreneurship Training Program for Undergraduate (No. 202210316091Y).

Acknowledgments

We thank all authors for their participation in this study and their insightful comments during the revision process.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Becker, W., and William, E. (2000). Teaching economics in the 21st century. J. Econ. Perspect. 14, 109–119. doi: 10.1257/jep.14.1.109

CrossRef Full Text | Google Scholar

Cashin, W. E., and Downey, R. G. (1992). Using global student rating items for summative evaluation. J. Educ. Psychol. 84, 563–572. doi: 10.1037/0022-0663.84.4.563

Chan, L. (2019). The practical dilemma and optimization suggestions of college students' evaluation of teaching. Educ. Modern. 6, 187–188. doi: 10.16541/j.cnki.2095-8420.2019.94.070

Chau, C. T. (1997). A bootstrap experiment on the statistical properties of students ratings of teaching effectiveness. Res. High. Educ. 38, 497–517. doi: 10.1023/A:1024918711471

Chen, Y., and Hoshower, L. B. (2003). Student evaluation of teaching effectiveness: an assessment of student perception and motivation. Assess. Eval. High. Educ. 28, 71–88. doi: 10.1080/02602930301683

Cheng, A. L., and Zhang, L. (2016). An empirical study on the reliability of college Students' teaching evaluation results. Heilongjiang High. Educ. Res. 7, 21–25. doi: 10.3969/j.issn.1003-2614.2016.07.005

Ching, G. (2019). A literature review on the student evaluation of teaching: an examination of the search, experience, and credence qualities of SET. High. Educ. Eval. Dev. 12, 63–84. doi: 10.1108/HEED-04-2018-0009

Clayson, D. E. (2018). Student evaluation of teaching and matters of reliability. Assess. Eval. High. Educ. 43, 666–681. doi: 10.1080/02602938.2017.1393495

Constantinou, C., and Wijnen-Meijer, M. (2022). Student evaluations of teaching and the development of a comprehensive measure of teaching effectiveness for medical schools. BMC Med. Educ. 22:113. doi: 10.1186/s12909-022-03148-6

De Neve, H. M. F. (1991). University teachers’ thinking about lecturing: student evaluation of lecturing as an improvement perspective for the lecturer. High. Educ. 22, 63–89. doi: 10.1007/BF02351200

Dennis, C. (2022). The student evaluation of teaching and likability: what the evaluations actually measure. Assess. Eval. High. Educ. 47, 313–326. doi: 10.1080/02602938.2021.1909702

Feistauer, D., and Richter, T. (2017). How reliable are students’ evaluations of teaching quality? A variance components approach. Assess. Eval. High. Educ. 42, 1263–1279. doi: 10.1080/02602938.2016.1261083

Gallagher, T. J. (2000). Embracing student evaluations of teaching: a case study. Teach. Sociol. 28, 140–147. doi: 10.1007/s11162-011-9229-0

Galbraith, C. S., Merrill, G. B., and Kline, D. M. (2012). Are student evaluations of teaching effectiveness valid for measuring student learning outcomes in business related classes? A neural network and bayesian analyses. Res. High. Educ. 53, 353–374. doi: 10.1007/s11162-011-9229-0

Gao, W. C., Bai, S. Y., and Yu, B. (2010). Research on the reliability of students' evaluation of teaching by using intra-group correlation coefficient. Teach. Manag. 3, 53–54. doi: 10.3969/j.issn.1004-5872.2010.01.023

Ginexi, E. M. (2003). General psychology course evaluations: differential survey response by expected grade. Teach. Psychol. 30, 248–251.

Google Scholar

Gu, R., Wang, H. N., and Lou, L. S. (2021). Optimization and application of data analysis strategy for college students' evaluation of teaching. J. Zhejiang Univ. Tech. 20, 201–207.

Hassanein, S., Abdrbo, A., and Ateeq, E. A. (2012). Student evaluation of Faculty at College of nursing. Inter. Conf. Manag. Educ. Innov. 37, 17–22.

He, Y. T. (2017). An analysis of Chinese college Students' evaluation of teaching from the perspective of ethics. J. Yangzhou Univ. 21, 29–33. doi: 10.19411/j.cnki.1007-8606.2017.03.005

Heckert, T. M., Latier, A., Ringwald-Burton, A., and Drazen, C. (2006). Relations among student effort, perceived class difficulty appropriateness, and student evaluations of teaching: is it possible to "buy" better evaluations through lenient grading? Coll. Stud. J. 40, 588–596.

Hocevar, M. D. (1991). Students' evaluations of teaching effectiveness: the stability of mean ratings of the same teachers over a 13-year period. Teach. Teach. Educ. 7, 303–314. doi: 10.1016/0742-051X(91)90001-6

Hong, X. B. (2010). Problems and countermeasures on the reliability and validity of teaching evaluation by college students. J. Ningbo Inst. Technol. 1:7. doi: 10.3969/j.issn.1008-7109.2010.01.025

Jiang, F., Guo, Y. F., Yang, Y. H., and Guo, Y. C. (2018). Rational choice to solve the dilemma of Students' teaching evaluation——constructing the "benefit sharing" evaluation system of schools, teachers and students. Contemp. Educ. For. 6, 66–73. doi: 10.13694/j.cnki.ddjylt.2018.06.009

Jiang, Y. J., and Lu, D. K. (2019). Research on the teaching evaluation system of first-class university students——taking ten universities including Harvard, Yale, Oxford, and Cambridge as examples. High. Educ. Explor. 49–54. doi: 10.3969/j.issn.1673-9760.2019.05.008

Jiang, H. C., and Xiong, Y. (2021). The indicators, characteristics and enlightenment of students' evaluation of teaching in Japanese national universities: taking education as an example. High. Educ. Res. 42, 103–109.

Kekale, J. (2000). Quality assessment in diverse disciplinary settings. High. Educ. 40, 465–488. doi: 10.1023/A:1004116205062

Kellaghan, T., and Stufflebeam, D. L. (2003). The CIPP model for evaluation. Springer Netherlands 4, 31–62. doi: 10.1007/978-94-010-0309-4

Khaola, P., and Thetsane, R. (2021). The validity and reliability of student evaluation of teaching at the National University of Lesotho. Int. J. Afr. High. Educ. 8, 139–157. doi: 10.6017/ijahe.v8i1.13367

Li, Z. G., Chen, Q., and Sun, T. T. (2019). "Student-centered" thinking and practice of improving students' teaching evaluation. Mod. Educ. Manag. 1, 62–66. doi: 10.16697/j.cnki.xdjygl.2019.01.011

Li, P. D., and Meng, Q. R. (2020). Why did the results of college students' evaluation of teaching fail: a study on influencing factors, adverse selection mechanism and its governance path. J. Educ. 2, 85–96. doi: 10.14082/j.cnki.1673-1298.2020.02.009

Li, C., Su, Y. J., and Ma, Y. C. (2017). Practical exploration of improving and perfecting students' teaching evaluation system in colleges and universities. Mod. Educ. Manag. 12, 69–73. doi: 10.3969/j.issn.1674-5485.2017.12.012

Liang, Y. C., Song, S. Q., and Zhao, A. J. (2020). Research on the alienation of the teaching evaluation system of college students. School Party Build. Ideol. Educ. 18, 55–57. doi: 10.19865/j.cnki.xxdj.2020.18.018

Long, Y. (2019). Empirical analysis of influencing factors of college teachers and Students' teaching evaluation scores. Bus. Acc. 6, 116–118. doi: 10.3969/j.issn.1002-5812.2019.06.038

Long, Y., and Wang, L. (2019). On the value dimension of "teaching academic" value of college students' evaluation of teaching. Contemp. Educ. Sci. 7–10. doi: 10.3969/j.issn.1672-2221.2019.08.003

Lv, P. (2014). The construction of student-centered teaching evaluation index system. Mod. Educ. Manag. 3, 42–45. doi: 10.3969/j.issn.1674-5485.2014.03.009

Marsh, H. W. (2007). “Students’ evaluations of university teaching: dimensionality, reliability, validity, potential biases and usefulness”, in The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective. eds. R. P. Perry and J. C. Smart (Dordrecht: Springer), 319–383.

Marsh, H., and Herbert, W. (1987). Students' evaluations of university teaching: research findings, methodological issues, and directions for future research. Int. J. Educ. Res. 11, 253–388. doi: 10.1016/0883-0355(87)90001-2

Morley, D. D. (2012). Claims about the reliability of student evaluations of instruction: the ecological fallacy rides again. Stud. Educ. Eval. 38, 15–20. doi: 10.1016/j.stueduc.2012.01.001

Pan, Y. H., and Zhang, Y. Y. (2016). An empirical study on the factors influencing the effectiveness of college Students' evaluation of teaching: a Student's perspective. Educ. Acad. Month. 7, 51–56. doi: 10.16477/j.cnki.issn1674-2311.2016.07.007

Richardson, J. T. E. (2005). Instruments for obtaining student feedback: a review of the literature. Assess. Eval. High. Educ. 30, 387–415. doi: 10.1080/02602930500099193

Schellhase, K. C. (2010). The relationship between student evaluation of instruction scores and faculty formal educational coursework. Athl. Train. Educ. J. 5, 156–164. doi: 10.4085/1947-380X-5.4.156

Spooren, P., Brockx, B., and Mortelmans, D. (2013). The validity of student evaluation of teaching. Rev. Educ. Res. 83, 598–642. doi: 10.3102/0034654313496870

Stroebe, W. (2020). Student evaluations of teaching encourages poor teaching and contributes to grade inflation: a theoretical and empirical analysis. Basic Appl. Soc. Psychol. 42, 276–294. doi: 10.1080/01973533.2020.1756817

Sun, Q. J. (2021). Existing problems and improvement strategies of college students' evaluation of teaching. Educ. Modern. 8, 126–129. doi: 10.12365/j.issn.2095-8420.2021.23.6816

Sun, A., and Sun, Y. Z. (2020). Game and reform in Students' teaching evaluation. High. Educ. Dev. Eval. 36, 47–156. doi: 10.3963/j.issn.1672-8742.2020.05.007

Svinicki, M. D. (2010). Encouraging your students to give feedback. New Dir. Teach. Learn. 2001, 17–24. doi: 10.1002/tl.24

Theall, M., Abrami, C., and Lisa, A. (2001). The student ratings debate: Are they valid ? how can we best use them. San Francisco, California: Jossey Bass Press.

Tsou, C. (2020). Student evaluation of teaching (SET): a critical review of the literature. Available at: https://www.researchgate.net/publication/339130475_Student_Evaluation_of_Teaching_SET_A_Critical_Review_of_the_Literature

Tu, Y. X., He, X. Q., and Rebiguli, A. (2019). On the evaluation of teaching by students in colleges and universities in my country. Educ. Modern. 32, 106–108. doi: 10.16541/j.cnki.2095-8420.2019.32.044

Uttl, B. (2021). Lessons learned from research on student evaluation of teaching in higher education. Stud. Feed. Teach. Sch. 13, 237–256. doi: 10.1007/978-3-030-75150-0_15

Wang, D. F., and Guan, L. (2017). Higher education quality evaluation from the perspective of students: theoretical construction and reflection. J. Nat. Inst. Educ. Admin. 5:75. doi: 10.3969/j.issn.1672-4038.2017.05.005

Wei, H. J., and Liu, M. (2013). A review of the research on "student evaluation of teaching" in colleges and universities. J. Nat. Inst. Educ. Admin. 1:62. doi: 10.3969/j.issn.1672-4038.2013.01.013

Wu, G. Y., Wang, C. Y., and Peng, X. F. (2015). Research on student-oriented classroom teaching evaluation index system in colleges and universities. Educ. Expl. 19–23.

Xu, C. T. (2017). Construction of college Students' teaching evaluation system based on the principle of continuous improvement: taking Jimei university as an example. J. Jimei Univ. 18, 70–75.

Yan, B. B., and Wei, T. L. (2016). Constructing a "student-centered" classroom teaching evaluation system in colleges and universities. J. Inner Mongolia Normal Univ. 29, 86–88.

Ye, L. (2000). Introduction to education . Beijing: People's Education Press.

Zhang, X. J., Liu, M. D., Qi, X. G., Liu, L. B., and Duan, X. X. (2019). Research on the optimization of student evaluation indicators in the new era of higher education. Voc. Tech. Educ. 2019:26. doi: 10.3969/j.issn.1008-3219.2019.26.012

Zhang, H. Y., Lu, S. Q., and Zhang, B. C. (2017). Research on the index system of learning evaluation and teaching based on the subject of students. Mod. Educ. Sci. 41–46. doi: 10.13980/j.cnki.xdjykx.201

Zhao, H. (2010). Evaluation of the classroom teaching of physical education with the CIPP pattern. J. Hebei Normal Univ. 12, 95–98. doi: 10.13763/j.cnki.jhebnu.ese.2010.12.023

Zhou, T. T. (2012). Analysis of the teaching evaluation index system of some foreign college students. China Univ. Teach. 2012:2. doi: 10.3969/j.issn.1005-0450.2012.02.028

Zhou, J. L., and Qin, Y. (2018). The basic types of college students' teaching evaluation behavior deviation and its relationship with students' background characteristics. Fudan Educ. Forum. 2018:6. doi: 10.13397/j.cnki.fef.2018.06.010

Keywords: students’ evaluation of teaching, review, comments, higher education, indicator systems

Citation: Zhao L, Xu P, Chen Y and Yan S (2022) A literature review of the research on students’ evaluation of teaching in higher education. Front. Psychol . 13:1004487. doi: 10.3389/fpsyg.2022.1004487

Received: 27 July 2022; Accepted: 29 August 2022; Published: 29 September 2022.

Reviewed by:

Copyright © 2022 Zhao, Xu, Chen and Yan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Shuangsheng Yan, [email protected]

† These authors have contributed equally to this work and share first authorship

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Literature Reviews

  • Getting Started
  • Choosing a Type of Review
  • Developing a Research Question
  • Searching the Literature
  • Searching Tips
  • ChatGPT [beta]
  • Documenting your Search
  • Using Citation Managers
  • Concept Mapping
  • Concept Map Definition

MindMeister

  • Writing the Review
  • Further Resources

Additional Tools

Google slides.

GSlides can create concept maps using their Diagram feature. Insert > Diagram > Hierarchy will give you some editable templates to use.

Tutorial on diagrams in GSlides .

MICROSOFT WORD

MS Word can create concept maps using Insert > SmartArt Graphic. Select Process, Cycle, Hierarchy, or Relationship to see templates.

NVivo  is software for qualitative analysis that has a concept map feature. Zotero libraries can be uploaded using ris files. NVivo Concept Map information.

A concept map or mind map is a visual representation of knowledge that illustrates relationships between concepts or ideas. It is a tool for organizing and representing information in a hierarchical and interconnected manner. At its core, a concept map consists of nodes, which represent individual concepts or ideas, and links, which depict the relationships between these concepts .

Below is a non-exhaustive list of tools that can facilitate the creation of concept maps.

research literature literature

www.canva.com

Canva is a user-friendly graphic design platform that enables individuals to create visual content quickly and easily. It offers a diverse array of customizable templates, design elements, and tools, making it accessible to users with varying levels of design experience. 

Pros: comes with many pre-made concept map templates to get you started

Cons : not all features are available in the free version

Explore Canva concept map templates here .

Note: Although Canva advertises an "education" option, this is for K-12 only and does not apply to university users.

research literature literature

www.lucidchart.com

Lucid has two tools that can create mind maps (what they're called inside Lucid): Lucidchart is the place to build, document, and diagram, and Lucidspark is the place to ideate, connect, and plan.

Lucidchart is a collaborative online diagramming and visualization tool that allows users to create a wide range of diagrams, including flowcharts, org charts, wireframes, and mind maps. Its mind-mapping feature provides a structured framework for brainstorming ideas, organizing thoughts, and visualizing relationships between concepts. 

Lucidspark , works as a virtual whiteboard. Here, you can add sticky notes, develop ideas through freehand drawing, and collaborate with your teammates. Has only one template for mind mapping.

Explore Lucid mind map creation here .

How to create mind maps using LucidSpark: 

Note: U-M students have access to Lucid through ITS. [ info here ] Choose the "Login w Google" option, use your @umich.edu account, and access should happen automatically.

research literature literature

www.figma.com

Figma is a cloud-based design tool that enables collaborative interface design and prototyping. It's widely used by UI/UX designers to create, prototype, and iterate on digital designs. Figma is the main design tool, and FigJam is their virtual whiteboard:

Figma  is a comprehensive design tool that enables designers to create and prototype high-fidelity designs

FigJam focuses on collaboration and brainstorming, providing a virtual whiteboard-like experience, best for concept maps

Explore FigJam concept maps here .

research literature literature

Note: There is a " Figma for Education " version for students that will provide access. Choose the "Login w Google" option, use your @umich.edu account, and access should happen automatically.

research literature literature

www.mindmeister.com

MindMeister  is an online mind mapping tool that allows users to visually organize their thoughts, ideas, and information in a structured and hierarchical format. It provides a digital canvas where users can create and manipulate nodes representing concepts or topics, and connect them with lines to show relationships and associations.

Features : collaborative, permits multiple co-authors, and multiple export formats. The free version allows up to 3 mind maps.

Explore  MindMeister templates here .

  • << Previous: Using Citation Managers
  • Next: Writing the Review >>
  • Last Updated: Feb 15, 2024 1:47 PM
  • URL: https://guides.lib.umich.edu/litreview
  • Research Guides

Grey Literature

  • What is Grey Literature? Activities
  • Why Use Grey Literature?
  • Types of Grey Literature

Sources of Grey Literature

  • Searching for Grey Literature
  • Evaluating Grey Literature Activity
  • How to Incorporate & Cite Grey Literature

This guide includes content adapted with permission from the University of Illinois Library .

research literature literature

Vanderbilt Resources for Grey Literature

  • Dissertations & Theses

The following are relevant databases the Vanderbilt Libraries are subscribed to.

Open to All (Free)

  • Dissertations & Theses Full Text This link opens in a new window Dissertations and theses from around the world in business, health & medicine, history, literature & language, science & technology, social sciences, and the arts. Includes Vanderbilt University dissertations and theses. more... less... Coverage: 1743-present.
  • EThOS: Electronic Theses Online Service This link opens in a new window UK’s national thesis service. National aggregated record of all doctoral theses awarded by UK Higher Education institutions. more... less... Coverage: 1800 - present.
  • Vanderbilt University Electronic Theses and Dissertations (ETD) This link opens in a new window Vanderbilt University theses and dissertations. more... less... Coverage: 1999 to present. User Limit: none.
  • Budget of the United States Government This link opens in a new window Budget message of the President, information about the President's budget proposals for a given fiscal year, and other budgetary publications. more... less... Coverage: Previous year, estimates for current year, projections for five years, and historical information from 1940.
  • CIA World Factbook This link opens in a new window Central Intelligence Agency's database: information on the history, people, government, economy, geography, communications, transportation, military, and transnational issues for countries, territories, and dependencies. more... less... Coverage: Current year.
  • CQ Congress Collection This link opens in a new window Public policy issues, information about members of Congress (1981-), vote analysis, and dictionary and encyclopedia-type entries on legislative terms and the legislative process.
  • EDGAR This link opens in a new window US government database of public company filings collected by the U.S. Securities and Exchange Commission. Full-text of company annual 10-K reports, quarterly 10-Q reports, proxy statements, prospectus, etc. more... less... Vendor Tutorials: http://www.sec.gov/edgar/quickedgar.htm
  • ERIC (ProQuest version) This link opens in a new window Journal articles, conferences, meetings, government documents, theses, dissertations, reports, audiovisual media, bibliographies, directories, books and monographs on education-related literature. more... less... Coverage: 1966 to present.
  • FAOSTAT This link opens in a new window Statistical database on food & agriculture. Includes production, trade, food balance sheets, fertilizer and pesticides, land use and irrigation, forest products, fishery products, population, agricultural machinery, and food aid shipments.
  • FDsys - Federal Digital System This link opens in a new window Government Printing Office's access to publications of the Federal Government.
  • GPO (Government Printing Office) Catalog, 1976- This link opens in a new window All types of US government documents, including Congressional reports, hearings, debates, and records; judiciary materials; and documents issued by executive departments (Defense, State, Labor, Office of the President, etc.).
  • Access World News Research Collection This link opens in a new window Research diverse perspectives, topics and trends that align with curricular areas such as Political Science, English, Sociology, Humanities, Business, International Studies and more. Features reliable, credible information from a wide variety of international, national and local news sources.
  • CQ Researcher This link opens in a new window Coverage of the most important issues of the day, written by experienced journalists, footnoted and professionally fact-checked. Articles include an overview, historical background, chronology, pro/con feature, and resources for additional research. more... less... Coverage: 1991 to present.
  • News (Gale OneFile) This link opens in a new window National, regional, and local newspapers; images; radio and television broadcasts and transcripts. more... less... Coverage: 1977 to present.
  • Newspapers.com Library Edition - Southeast Collection This link opens in a new window Historical newspapers archive containing thousands of regional, state, and small local newspapers from the southeastern U.S.
  • ProQuest News and Newspapers This link opens in a new window Direct link to the newspaper section of ProQuest. Includes Alt-Press Watch, Ethnic NewsWatch, GenderWatch, Latin American Newsstand, Canadian Newsstand Complete, ProQuest Newsstand, ProQuest Historical Newspapers. Chicago Tribune, Christian Science Monitor, Los Angeles Times, New York Times, Wall Street Journal, Washington Post, Times of London, and The Tennessean. more... less... All current Vanderbilt University students, faculty, and staff have access, both on and off campus.
  • Tennessean (2002-present) This link opens in a new window Nashville's local and regional newspaper: The Tennessean. more... less... Coverage: Nov. 7, 2002, to date, with incomplete coverage back to Jan. 1, 1999.
  • ProQuest Statistical Insight This link opens in a new window Federal, state, and international statistical, economic, demographic, and geographic information. National income and production statistics, retail and wholesale trade data, interest rates, economic indicators, and social and demographic statistics, plus data on energy, environmental protection, agriculture, fisheries, crime, etc. more... less... Coverage: From the U.S. government, 1973 to present; from international intergovernmental, 1983 to present.
  • Statista This link opens in a new window Quantitative data, statistics, infographics, and related information compiled from market research institutes, trade publications, scientific journals, and government databases.
  • Statistical Abstract of the United States (ProQuest) This link opens in a new window Extensive selection of statistics for the United States, with selected data for regions, divisions, states, metropolitan areas, cities, and foreign countries from reports and records of government and private agencies. more... less... Access Note: If prompted to select an account, select Vanderbilt University.
  • Global Think Tanks This link opens in a new window Curated, high quality policy reports, briefs, analyses, working papers, and datasets produced by the world’s leading policy experts, think tanks, research centers, IGOs, and NGOs.
  • Nexis Uni (formerly LexisNexis Academic) This link opens in a new window News and newspaper stories, federal and state court cases, US law reviews and journals, company profiles and directories. Company Dossier; data on public figures. more... less... Access Note: Personal account required for some functionality. User Limit: Unlimited.
  • PAIS International This link opens in a new window Public Affairs Information Service: journal articles, books, government documents, statistical directories, grey literature, research reports, conference papers, web content, etc. covering issues in the public debate. more... less... Coverage: PAIS International is the current file covering 1972 to present, and PAIS Archive includes the content from printed volumes published 1915-1976. User Limit: Unlimited.
  • ProQuest Central This link opens in a new window Scholarly journals, newspapers, dissertations, and business information. Includes: ABI/INFORM Complete, Accounting & Tax, Banking Information Source, Canadian Newsstand Complete, CBCA Complete, Hoover's Company Profiles, OxResearch, Pharmaceutical News Index, Asian Business & Reference, Biology Journals, Career and Technical Education, Computing, Criminal Justice, Education Journals, European Business, Family Health, Health & Medical Complete, Health Management, Military Collection, Newsstand, Nursing & Allied Health Source, Psychology Journals, Religion, Research Library, Science Journals, Social Science Journals, Telecommunications, Snapshots. more... less... All current Vanderbilt University students, faculty, and staff have access, both on and off campus.
  • ProQuest Social Sciences Premium Collection This link opens in a new window International literature in the social sciences: politics, sociology, social services, anthropology, criminology and education. IBSS, PAIS and Sociological Abstracts. Abstracts, indexing and full text of articles, books, chapters, dissertations, working papers. • Applied Social Sciences Index and Abstracts (ASSIA) • Criminal Justice Database • Education Database • ERIC • International Bibliography of the Social Sciences (IBSS) • Linguistics and Language Behavior Abstracts (LLBA) • PAIS Index • Political Science Database • PTSDpubs • Social Science Database • Social Services Abstracts • Sociological Abstracts • Sociology Database • Worldwide Political Science Abstracts
  • Web of Science This link opens in a new window Citation database from journals and international conference proceedings. Cross-search of the Web of Science Core Collection, BIOSIS Previews, and the SciELO Citation Index (Latin America, Spain, Portugal, the Caribbean and South Africa). more... less... All current Vanderbilt University students, faculty, and staff have access, both on and off campus.

Tools for Searching for Grey Literature

  • Directories
  • Custom Google Search Bars
  • General Sources
  • FIND POLICY A side project of Transparify, search think tank sites grouped by topic and location.
  • Open Think Tank Directory A global collection of 2700 think tanks and related organizations.
  • GreySource GreySource provides examples of grey literature to the average net-user and in so doing profiles organizations responsible for its production and/or processing. GreySource identifies the hyperlink directly embedded in a resource, thus allowing immediate and virtual exposure to grey literature.
  • Grey Guide portal and repository for resources in the field of grey literature.
  • OpenGrey A search portal for European grey literature. It covers Science, Technology, Biomedical Science, Economics, Social Science and the Humanities.
  • World Association of Non-Governmental Organizations (WANGO) Worldwide NGO Directory Search for NGO's based on location.
  • OpenDOAR OpenDOAR is the quality-assured, global Directory of Open Access Repositories. You can search and browse through thousands of registered repositories based on a range of features, such as location, software or type of material held
  • UNESCO Database of Non-Governmental Organizations (NGOs) Contains the profiles of non-governmental organizations that have official relations with UNESCO.
  • UN NGO Branch - Integrated Civil Society Organizations System The integrated Civil Society Organizations (iCSO) System, developed by the Department of Economic and Social Affairs (DESA), facilitates interactions between civil society organizations and DESA. The system provides online registration of general profiles for civil society organizations, including address, contacts, activities and meeting participation.
  • Harvard Kennedy School of Government Think Tank Search A Google Custom Search of more than 1200 think tanks and research centers.
  • Non-Governmental Organizations Custom searches to find information from NGOs created by Government Documents Round Table (GODORT)
  • JSTOR Open Research Reports JSTOR hosts a growing curated collection of more than 45,000 open research reports from 175 policy institutes from around the world. These publications are freely accessible to everyone on JSTOR and discoverable as their own content type alongside journals, books, and primary sources. We update research reports on our platform each month as they become available through contributing institutes.Research reports provide current analysis on many of today’s most discussed and debated issues from a diversity of ideological and international perspectives representing 39 countries and 29 languages. A sample of topics would include climate change, border security, fake news, cybersecurity, electric vehicles, artificial intelligence, energy policy, gender issues, terrorism, remote learning, and various public health issues, including COVID-19.
  • Government Documents
  • Non-Scholarly Academic
  • GovInfo GovInfo is a service of the United States Government Publishing Office (GPO), which is a Federal agency in the legislative branch. GovInfo provides free public access to official publications from all three branches of the Federal Government.
  • United Nations Digital Library The online catalog of UN documents and publications. The Digital Library includes UN documents, voting data, speeches, maps, and open access publications. The platform provides access to UN-produced materials in digital format and bibliographic records for print UN documents starting in 1979. System features include linked data between related documentation such as resolutions, meeting records and voting, and refining of searches by UN body, agency or type of document.
  • Social Science Research Network (SSRN) SSRN, formerly known as Social Science Research Network, is an open access research platform used to share early-stage research, evolve ideas, measure results, and connect scholars around the world.

Grey Literature by Topic

  • Public Health
  • Governance and Conflict
  • Migration and Culture
  • Science, Technology and Enviroment
  • Development and Aid
  • The Community Health Online Digital Archive & Research Resource (CHODARR) An archived virtual library of community-based and government publications related to health and social welfare policy, including grey literature, with emphasis on Vancouver-area resources and organizations. Note: This source is no longer being updated. You can browse the individual collections, or use Google's site search function to search across it. Example search in Google: site:chodarr.org AIDS
  • PolicyArchive PolicyArchive is a comprehensive digital library of public policy research containing over 30,000 documents.
  • Migration Policy Institute Reports, fact sheets, policy briefs, articles, commentaries and more on several topics related to Migration.
  • National Technical Reports Library The National Technical Information Service acquires, indexes, abstracts, and archives the largest collection of U.S. government-sponsored technical reports in existence. The NTRL offers online, free and open access to these authenticated government technical reports
  • Science.gov Science.gov searches over 60 databases and over 2,200 scientific websites to provide users with access to more than 200 million pages of authoritative federal science information including research and development results.
  • USAID Development Experience Clearinghouse Search 50 years worth of USAID project records and other related documents on a variety of topics.
  • The World Bank Research and Publications Data, research and reports on topics related to poverty and development.
  • Eldis Eldis supports free and open access to useful and relevant research on global development challenges.
  • << Previous: Types of Grey Literature
  • Next: Searching for Grey Literature >>
  • Last Updated: Feb 14, 2024 11:35 AM
  • URL: https://researchguides.library.vanderbilt.edu/greyliterature

Creative Commons License

research literature literature

AI Literature Review Generator

Automated literature review creation tool.

  • Academic Research: Create a literature review for your thesis, dissertation, or research paper.
  • Professional Research: Conduct a literature review for a project, report, or proposal at work.
  • Content Creation: Write a literature review for a blog post, article, or book.
  • Personal Research: Conduct a literature review to deepen your understanding of a topic of interest.

New & Trending Tools

Speech maker ai, ai letter writer, movie suggestion generator.

research literature literature

  • Research Guides
  • Topic Guides

Psychology Literature Searching

Information literacy in psychology.

  • Introductory Level
  • Research Methods
  • Getting ideas - a search example
  • How to find empirical research
  • Finding related research through citations

Trust and reliability are foundational to research scholarship.

  • Choosing selective search tools can help you narrow the danger of you encountering dubious claims.
  • Following citation chains is another way of finding sources other researchers rely on. See the guide for Citation Chasing .

A major distinction in the area of Information Literacy is the difference between misinformation and disinformation . (See Muhammed T, S., & Mathew, S. K. (2022). The disaster of misinformation: A review of research in social media . International Journal of Data Science and Analytics , 13 (4), 271–285. https://doi.org/10.1007/s41060-022-00311-6)

If a claim is stated or made without a source or a citation, it has the potential of being either misinformation or disinformation.

The difference hinges on intent. Misinformation is wrong information passed along with no intention to mislead. Misinformation if it is caught and corrected can lead to the growth of knowledge. But it can be hard to free yourself from misinformation if it fits what you want to see (See Ecker, U.K.H., Lewandowsky, S., Cook, J. et al. The psychological drivers of misinformation belief and its resistance to correction . Nat Rev Psychol 1, 13–29 (2022). https://doi.org/10.1038/s44159-021-00006-y )

Lateral reading - which means checking sources ( citation chasing ) for their credibility can help. Also keep asking yourself why you believe or want to believe statements and conclusions.

Disinformation is intentional and most often creeps into psychological research in the form of data fabrication .

Published research articles can be wrong in their conclusions without intent. Because of the peer review process , disinformation is less common in published academic scholarship. But it still does happen with falsified research and can be hard to detect. The site Retraction Watch keeps track of sources that have been identified as problematic. If you are basing your claims on one or a few key research papers, it is a good idea to see if they are suspect.

For a psychological research approach to information literacy, see the journal article in Current Directions in Science Critical Ignoring as a Core Competence for Digital Citizens and/or a friendlier introduction to it in the NiemanLab article What’s the best way to deal with a flood of misinformation? Maybe it’s time for some deliberate ignorance

The Temple Library tutorial How to SIFT through Misinformation reviews general best practices

  • << Previous: Finding related research through citations
  • Last Updated: Feb 14, 2024 8:15 AM
  • URL: https://guides.temple.edu/psychology_levels

Temple University

University libraries.

See all library locations

  • Library Directory
  • Locations and Directions
  • Frequently Called Numbers

Twitter Icon

Need help? Email us at [email protected]

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Advanced Search
  • Journal List
  • J Med Libr Assoc
  • v.106(4); 2018 Oct

A systematic approach to searching: an efficient and complete method to develop literature searches

Associated data.

Creating search strategies for systematic reviews, finding the best balance between sensitivity and specificity, and translating search strategies between databases is challenging. Several methods describe standards for systematic search strategies, but a consistent approach for creating an exhaustive search strategy has not yet been fully described in enough detail to be fully replicable. The authors have established a method that describes step by step the process of developing a systematic search strategy as needed in the systematic review. This method describes how single-line search strategies can be prepared in a text document by typing search syntax (such as field codes, parentheses, and Boolean operators) before copying and pasting search terms (keywords and free-text synonyms) that are found in the thesaurus. To help ensure term completeness, we developed a novel optimization technique that is mainly based on comparing the results retrieved by thesaurus terms with those retrieved by the free-text search words to identify potentially relevant candidate search terms. Macros in Microsoft Word have been developed to convert syntaxes between databases and interfaces almost automatically. This method helps information specialists in developing librarian-mediated searches for systematic reviews as well as medical and health care practitioners who are searching for evidence to answer clinical questions. The described method can be used to create complex and comprehensive search strategies for different databases and interfaces, such as those that are needed when searching for relevant references for systematic reviews, and will assist both information specialists and practitioners when they are searching the biomedical literature.

INTRODUCTION

Librarians and information specialists are often involved in the process of preparing and completing systematic reviews (SRs), where one of their main tasks is to identify relevant references to include in the review [ 1 ]. Although several recommendations for the process of searching have been published [ 2 – 6 ], none describe the development of a systematic search strategy from start to finish.

Traditional methods of SR search strategy development and execution are highly time consuming, reportedly requiring up to 100 hours or more [ 7 , 8 ]. The authors wanted to develop systematic and exhaustive search strategies more efficiently, while preserving the high sensitivity that SR search strategies necessitate. In this article, we describe the method developed at Erasmus University Medical Center (MC) and demonstrate its use through an example search. The efficiency of the search method and outcome of 73 searches that have resulted in published reviews are described in a separate article [ 9 ].

As we aimed to describe the creation of systematic searches in full detail, the method starts at a basic level with the analysis of the research question and the creation of search terms. Readers who are new to SR searching are advised to follow all steps described. More experienced searchers can consider the basic steps to be existing knowledge that will already be part of their normal workflow, although step 4 probably differs from general practice. Experienced searchers will gain the most from reading about the novelties in the method as described in steps 10–13 and comparing the examples given in the supplementary appendix to their own practice.

CREATING A SYSTEMATIC SEARCH STRATEGY

Our methodology for planning and creating a multi-database search strategy consists of the following steps:

  • Determine a clear and focused question
  • Describe the articles that can answer the question
  • Decide which key concepts address the different elements of the question
  • Decide which elements should be used for the best results
  • Choose an appropriate database and interface to start with
  • Document the search process in a text document
  • Identify appropriate index terms in the thesaurus of the first database
  • Identify synonyms in the thesaurus
  • Add variations in search terms
  • Use database-appropriate syntax, with parentheses, Boolean operators, and field codes
  • Optimize the search
  • Evaluate the initial results
  • Check for errors
  • Translate to other databases
  • Test and reiterate

Each step in the process is reflected by an example search described in the supplementary appendix .

1. Determine a clear and focused question

A systematic search can best be applied to a well-defined and precise research or clinical question. Questions that are too broad or too vague cannot be answered easily in a systematic way and will generally result in an overwhelming number of search results. On the other hand, a question that is too specific will result into too few or even zero search results. Various papers describe this process in more detail [ 10 – 12 ].

2. Describe the articles that can answer the question

Although not all clinical or research questions can be answered in the literature, the next step is to presume that the answer can indeed be found in published studies. A good starting point for a search is hypothesizing what the research that can answer the question would look like. These hypothetical (when possible, combined with known) articles can be used as guidance for constructing the search strategy.

3. Decide which key concepts address the different elements of the question

Key concepts are the topics or components that the desired articles should address, such as diseases or conditions, actions, substances, settings, domains (e.g., therapy, diagnosis, etiology), or study types. Key concepts from the research question can be grouped to create elements in the search strategy.

Elements in a search strategy do not necessarily follow the patient, intervention, comparison, outcome (PICO) structure or any other related structure. Using the PICO or another similar framework as guidance can be helpful to consider, especially in the inclusion and exclusion review stage of the SR, but this is not necessary for good search strategy development [ 13 – 15 ]. Sometimes concepts from different parts of the PICO structure can be grouped together into one search element, such as when the desired outcome is frequently described in a certain study type.

4. Decide which elements should be used for the best results

Not all elements of a research question should necessarily be used in the search strategy. Some elements are less important than others or may unnecessarily complicate or restrict a search strategy. Adding an element to a search strategy increases the chance of missing relevant references. Therefore, the number of elements in a search strategy should remain as low as possible to optimize recall.

Using the schema in Figure 1 , elements can be ordered by their specificity and importance to determine the best search approach. Whether an element is more specific or more general can be measured objectively by the number of hits retrieved in a database when searching for a key term representing that element. Depending on the research question, certain elements are more important than others. If articles (hypothetically or known) exist that can answer the question but lack a certain element in their titles, abstracts, or keywords, that element is unimportant to the question. An element can also be unimportant because of expected bias or an overlap with another element.

An external file that holds a picture, illustration, etc.
Object name is jmla-106-531-f001.jpg

Schema for determining the optimal order of elements

Bias in elements

The choice of elements in a search strategy can introduce bias through use of overly specific terminology or terms often associated with positive outcomes. For the question “does prolonged breastfeeding improve intelligence outcomes in children?,” searching specifically for the element of duration will introduce bias, as articles that find a positive effect of prolonged breastfeeding will be much more likely to mention time factors in their titles or abstracts.

Overlapping elements

Elements in a question sometimes overlap in their meaning. Sometimes certain therapies are interventions for one specific disease. The Lichtenstein technique, for example, is a repair method for inguinal hernias. There is no need to include an element of “inguinal hernias” to a search for the effectiveness of the Lichtenstein therapy. Likewise, sometimes certain diseases are only found in certain populations. Adding such an overlapping element could lead to missing relevant references.

The elements to use in a search strategy can be found in the plot of elements in Figure 1 , by following the top row from left to right. For this method, we recommend starting with the most important and specific elements. Then, continue with more general and important elements until the number of results is acceptable for screening. Determining how many results are acceptable for screening is often a matter of negotiation with the SR team.

5. Choose an appropriate database and interface to start with

Important factors for choosing databases to use are the coverage and the presence of a thesaurus. For medically oriented searches, the coverage and recall of Embase, which includes the MEDLINE database, are superior to those of MEDLINE [ 16 ]. Each of these two databases has its own thesaurus with its own unique definitions and structure. Because of the complexity of the Embase thesaurus, Emtree, which contains much more specific thesaurus terms than the MEDLINE Medical Subject Headings (MeSH) thesaurus, translation from Emtree to MeSH is easier than the other way around. Therefore, we recommend starting in Embase.

MEDLINE and Embase are available through many different vendors and interfaces. The choice of an interface and primary database is often determined by the searcher’s accessibility. For our method, an interface that allows searching with proximity operators is desirable, and full functionality of the thesaurus, including explosion of narrower terms, is crucial. We recommend developing a personal workflow that always starts with one specific database and interface.

6. Document the search process in a text document

We advise designing and creating the complete search strategies in a log document, instead of directly in the database itself, to register the steps taken and to make searches accountable and reproducible. The developed search strategies can be copied and pasted into the desired databases from the log document. This way, the searcher is in control of the whole process. Any change to the search strategy should be done in the log document, assuring that the search strategy in the log is always the most recent.

7. Identify appropriate index terms in the thesaurus of the first database

Searches should start by identifying appropriate thesaurus terms for the desired elements. The thesaurus of the database is searched for matching index terms for each key concept. We advise restricting the initial terms to the most important and most relevant terms. Later in the process, more general terms can be added in the optimization process, in which the effect on the number of hits, and thus the desirability of adding these terms, can be evaluated more easily.

Several factors can complicate the identification of thesaurus terms. Sometimes, one thesaurus term is found that exactly describes a specific element. In contrast, especially in more general elements, multiple thesaurus terms can be found to describe one element. If no relevant thesaurus terms have been found for an element, free-text terms can be used, and possible thesaurus terms found in the resulting references can be added later (step 11).

Sometimes, no distinct thesaurus term is available for a specific key concept that describes the concept in enough detail. In Emtree, one thesaurus term often combines two or more elements. The easiest solution for combining these terms for a sensitive search is to use such a thesaurus term in all elements where it is relevant. Examples are given in the supplementary appendix .

8. Identify synonyms in the thesaurus

Most thesauri offer a list of synonyms on their term details page (named Synonyms in Emtree and Entry Terms in MeSH). To create a sensitive search strategy for SRs, these terms need to be searched as free-text keywords in the title and abstract fields, in addition to searching their associated thesaurus terms.

The Emtree thesaurus contains more synonyms (300,000) than MeSH does (220,000) [ 17 ]. The difference in number of terms is even higher considering that many synonyms in MeSH are permuted terms (i.e., inversions of phrases using commas).

Thesaurus terms are ordered in a tree structure. When searching for a more general thesaurus term, the more specific (narrower) terms in the branches below that term will also be searched (this is frequently referred to as “exploding” a thesaurus term). However, to perform a sensitive search, all relevant variations of the narrower terms must be searched as free-text keywords in the title or abstract, in addition to relying on the exploded thesaurus term. Thus, all articles that describe a certain narrower topic in their titles and abstracts will already be retrieved before MeSH terms are added.

9. Add variations in search terms (e.g., truncation, spelling differences, abbreviations, opposites)

Truncation allows a searcher to search for words beginning with the same word stem. A search for therap* will, thus, retrieve therapy, therapies, therapeutic, and all other words starting with “therap.” Do not truncate a word stem that is too short. Also, limitations of interfaces should be taken into account, especially in PubMed, where the number of search term variations that can be found by truncation is limited to 600.

Databases contain references to articles using both standard British and American English spellings. Both need to be searched as free-text terms in the title and abstract. Alternatively, many interfaces offer a certain code to replace zero or one characters, allowing a search for “pediatric” or “paediatric” as “p?ediatric.” Table 1 provides a detailed description of the syntax for different interfaces.

Field codes in five most used interfaces for biomedical literature searching

Searching for abbreviations can identify extra, relevant references and retrieve more irrelevant ones. The search can be more focused by combining the abbreviation with an important word that is relevant to its meaning or by using the Boolean “NOT” to exclude frequently observed, clearly irrelevant results. We advise that searchers do not exclude all possible irrelevant meanings, as it is very time consuming to identify all the variations, it will result in unnecessarily complicated search strategies, and it may lead to erroneously narrowing the search and, thereby, reduce recall.

Searching partial abbreviations can be useful for retrieving relevant references. For example, it is very likely that an article would mention osteoarthritis (OA) early in the abstract, replacing all further occurrences of osteoarthritis with OA . Therefore, it may not contain the phrase “hip osteoarthritis” but only “hip oa.”

It is also important to search for the opposites of search terms to avoid bias. When searching for “disease recurrence,” articles about “disease free” may be relevant as well. When the desired outcome is survival , articles about mortality may be relevant.

10. Use database-appropriate syntax, with parentheses, Boolean operators, and field codes

Different interfaces require different syntaxes, the special set of rules and symbols unique to each database that define how a correctly constructed search operates. Common syntax components include the use of parentheses and Boolean operators such as “AND,” “OR,” and “NOT,” which are available in all major interfaces. An overview of different syntaxes for four major interfaces for bibliographic medical databases (PubMed, Ovid, EBSCOhost, Embase.com, and ProQuest) is shown in Table 1 .

Creating the appropriate syntax for each database, in combination with the selected terms as described in steps 7–9, can be challenging. Following the method outlined below simplifies the process:

  • Create single-line queries in a text document (not combining multiple record sets), which allows immediate checking of the relevance of retrieved references and efficient optimization.
  • Type the syntax (Boolean operators, parentheses, and field codes) before adding terms, which reduces the chance that errors are made in the syntax, especially in the number of parentheses.
  • Use predefined proximity structures including parentheses, such as (() ADJ3 ()) in Ovid, that can be reused in the query when necessary.
  • Use thesaurus terms separately from free-text terms of each element. Start an element with all thesaurus terms (using “OR”) and follow with the free-text terms. This allows the unique optimization methods as described in step 11.
  • When adding terms to an existing search strategy, pay close attention to the position of the cursor. Make sure to place it appropriately either in the thesaurus terms section, in the title/abstract section, or as an addition (broadening) to an existing proximity search.

The supplementary appendix explains the method of building a query in more detail, step by step for different interfaces: PubMed, Ovid, EBSCOhost, Embase.com, and ProQuest. This method results in a basic search strategy designed to retrieve some relevant references upon which a more thorough search strategy can be built with optimization such as described in step 11.

11. Optimize the search

The most important question when performing a systematic search is whether all (or most) potentially relevant articles have been retrieved by the search strategy. This is also the most difficult question to answer, since it is unknown which and how many articles are relevant. It is, therefore, wise first to broaden the initial search strategy, making the search more sensitive, and then check if new relevant articles are found by comparing the set results (i.e., search for Strategy #2 NOT Strategy #1 to see the unique results).

A search strategy should be tested for completeness. Therefore, it is necessary to identify extra, possibly relevant search terms and add them to the test search in an OR relationship with the already used search terms. A good place to start, and a well-known strategy, is scanning the top retrieved articles when sorted by relevance, looking for additional relevant synonyms that could be added to the search strategy.

We have developed a unique optimization method that has not been described before in the literature. This method often adds valuable extra terms to our search strategy and, therefore, extra, relevant references to our search results. Extra synonyms can be found in articles that have been assigned a certain set of thesaurus terms but that lack synonyms in the title and/or abstract that are already present in the current search strategy. Searching for thesaurus terms NOT free-text terms will help identify missed free-text terms in the title or abstract. Searching for free-text terms NOT thesaurus terms will help identify missed thesaurus terms. If this is done repeatedly for each element, leaving the rest of the query unchanged, this method will help add numerous relevant terms to the query. These steps are explained in detail for five different search platforms in the supplementary appendix .

12. Evaluate the initial results

The results should now contain relevant references. If the interface allows relevance ranking, use that in the evaluation. If you know some relevant references that should be included in the research, search for those references specifically; for example, combine a specific (first) author name with a page number and the publication year. Check whether those references are retrieved by the search. If the known relevant references are not retrieved by the search, adapt the search so that they are. If it is unclear which element should be adapted to retrieve a certain article, combine that article with each element separately.

Different outcomes are desired for different types of research questions. For instance, in the case of clinical question answering, the researcher will not be satisfied with many references that contain a lot of irrelevant references. A clinical search should be rather specific and is allowed to miss a relevant reference. In the case of an SR, the researchers do not want to miss any relevant reference and are willing to handle many irrelevant references to do so. The search for references to include in an SR should be very sensitive: no included reference should be missed. A search that is too specific or too sensitive for the intended goal can be adapted to become more sensitive or specific. Steps to increase sensitivity or specificity of a search strategy can be found in the supplementary appendix .

13. Check for errors

Errors might not be easily detected. Sometimes clues can be found in the number of results, either when the number of results is much higher or lower than expected or when many retrieved references are not relevant. However, the number expected is often unknown, and very sensitive search strategies will always retrieve many irrelevant articles. Each query should, therefore, be checked for errors.

One of the most frequently occurring errors is missing the Boolean operator “OR.” When no “OR” is added between two search terms, many interfaces automatically add an “AND,” which unintentionally reduces the number of results and likely misses relevant references. One good strategy to identify missing “OR”s is to go to the web page containing the full search strategy, as translated by the database, and using Ctrl-F search for “AND.” Check whether the occurrences of the “AND” operator are deliberate.

Ideally, search strategies should be checked by other information specialists [ 18 ]. The Peer Review of Electronic Search Strategies (PRESS) checklist offers good guidance for this process [ 4 ]. Apart from the syntax (especially Boolean operators and field codes) of the search strategy, it is wise to have the search terms checked by the clinician or researcher familiar with the topic. At Erasmus MC, researchers and clinicians are involved during the complete process of structuring and optimizing the search strategy. Each word is added after the combined decision of the searcher and the researcher, with the possibility of directly comparing results with and without the new term.

14. Translate to other databases

To retrieve as many relevant references as possible, one has to search multiple databases. Translation of complex and exhaustive queries between different databases can be very time consuming and cumbersome. The single-line search strategy approach detailed above allows quick translations using the find and replace method in Microsoft Word (<Ctrl-H>).

At Erasmus MC, macros based on the find-and-replace method in Microsoft Word have been developed for easy and fast translation between the most used databases for biomedical and health sciences questions. The schema that is followed for the translation between databases is shown in Figure 2 . Most databases simply follow the structure set by the Embase.com search strategy. The translation from Emtree terms to MeSH terms for MEDLINE in Ovid often identifies new terms that need to be added to the Embase.com search strategy before the translation to other databases.

An external file that holds a picture, illustration, etc.
Object name is jmla-106-531-f002.jpg

Schematic representation of translation between databases used at Erasmus University Medical Center

Dotted lines represent databases that are used in less than 80% of the searches.

Using five different macros, a thoroughly optimized query in Embase.com can be relatively quickly translated into eight major databases. Basic search strategies will be created to use in many, mostly smaller, databases, because such niche databases often do not have extensive thesauri or advanced syntax options. Also, there is not much need to use extensive syntax because the number of hits and, therefore, the amount of noise in these databases is generally low. In MEDLINE (Ovid), PsycINFO (Ovid), and CINAHL (EBSCOhost), the thesaurus terms must be adapted manually, as each database has its own custom thesaurus. These macros and instructions for their installation, use, and adaptation are available at bit.ly/databasemacros.

15. Test and reiterate

Ideally, exhaustive search strategies should retrieve all references that are covered in a specific database. For SR search strategies, checking searches for their recall is advised. This can be done after included references have been determined by the authors of the systematic review. If additional papers have been identified through other non-database methods (i.e., checking references in included studies), results that were not identified by the database searches should be examined. If these results were available in the databases but not located by the search strategy, the search strategy should be adapted to try to retrieve these results, as they may contain terms that were omitted in the original search strategies. This may enable the identification of additional relevant results.

A methodology for creating exhaustive search strategies has been created that describes all steps of the search process, starting with a question and resulting in thorough search strategies in multiple databases. Many of the steps described are not new, but together, they form a strong method creating high-quality, robust searches in a relatively short time frame.

Our methodology is intended to create thoroughness for literature searches. The optimization method, as described in step 11, will identify missed synonyms or thesaurus terms, unlike any other method that largely depends on predetermined keywords and synonyms. Using this method results in a much quicker search process, compared to traditional methods, especially because of the easier translation between databases and interfaces (step 13). The method is not a guarantee for speed, since speed depends on many factors, including experience. However, by following the steps and using the tools as described above, searchers can gain confidence first and increase speed through practice.

What is new?

This method encourages searchers to start their search development process using empty syntax first and later adding the thesaurus terms and free-text synonyms. We feel this helps the searcher to focus on the search terms, instead of on the structure of the search query. The optimization method in which new terms are found in the already retrieved articles is used in some other institutes as well but has to our knowledge not been described in the literature. The macros to translate search strategies between interfaces are unique in this method.

What is different compared to common practice?

Traditionally, librarians and information specialists have focused on creating complex, multi-line (also called line-by-line) search strategies, consisting of multiple record sets, and this method is frequently advised in the literature and handbooks [ 2 , 19 – 21 ]. Our method, instead, uses single-line searches, which is critical to its success. Single-line search strategies can be easily adapted by adding or dropping a term without having to recode numbers of record sets, which would be necessary in multi-line searches. They can easily be saved in a text document and repeated by copying and pasting for search updates. Single-line search strategies also allow easy translation to other syntaxes using find-and-replace technology to update field codes and other syntax elements or using macros (step 13).

When constructing a search strategy, the searcher might experience that certain parentheses in the syntax are unnecessary, such as parentheses around all search terms in the title/abstract portion, if there is only one such term, there are double parentheses in the proximity statement, or one of the word groups exists for only one word. One might be tempted to omit those parentheses for ease of reading and management. However, during the optimization process, the searcher is likely to find extra synonyms that might consist of one word. To add those terms to the first query (with reduced parentheses) requires adding extra parentheses (meticulously placing and counting them), whereas, in the latter search, it only requires proper placement of those terms.

Many search methods highly depend on the PICO framework. Research states that often PICO or PICOS is not suitable for every question [ 22 , 23 ]. There are other acronyms than PICO—such as sample, phenomenon of interest, design, evaluation, research type (SPIDER) [ 24 ]—but each is just a variant. In our method, the most important and specific elements of a question are being analyzed for building the best search strategy.

Though it is generally recommended that searchers search both MEDLINE and Embase, most use MEDLINE as the starting point. It is considered the gold standard for biomedical searching, partially due to historical reasons, since it was the first of its kind, and more so now that it is freely available via the PubMed interface. Our method can be used with any database as a starting point, but we use Embase instead of MEDLINE or another database for a number of reasons. First, Embase provides both unique content and the complete content of MEDLINE. Therefore, searching Embase will be, by definition, more complete than searching MEDLINE only. Second, the number of terms in Emtree (the Embase thesaurus) is three times as high as that of MeSH (the MEDLINE thesaurus). It is easier to find MeSH terms after all relevant Emtree terms have been identified than to start with MeSH and translate to Emtree.

At Erasmus MC, the researchers sit next to the information specialist during most of the search strategy design process. This way, the researchers can deliver immediate feedback on the relevance of proposed search terms and retrieved references. The search team then combines knowledge about databases with knowledge about the research topic, which is an important condition to create the highest quality searches.

Limitations of the method

One disadvantage of single-line searches compared to multi-line search strategies is that errors are harder to recognize. However, with the methods for optimization as described (step 11), errors are recognized easily because missed synonyms and spelling errors will be identified during the process. Also problematic is that more parentheses are needed, making it more difficult for the searcher and others to assess the logic of the search strategy. However, as parentheses and field codes are typed before the search terms are added (step 10), errors in parentheses can be prevented.

Our methodology works best if used in an interface that allows proximity searching. It is recommended that searchers with access to an interface with proximity searching capabilities select one of those as the initial database to develop and optimize the search strategy. Because the PubMed interface does not allow proximity searches, phrases or Boolean “AND” combinations are required. Phrase searching complicates the process and is more specific, with the higher risk of missing relevant articles, and using Boolean “AND” combinations increases sensitivity but at an often high loss of specificity. Due to some searchers’ lack of access to expensive databases or interfaces, the freely available PubMed interface may be necessary to use, though it should never be the sole database used for an SR [ 2 , 16 , 25 ]. A limitation of our method is that it works best with subscription-based and licensed resources.

Another limitation is the customization of the macros to a specific institution’s resources. The macros for the translation between different database interfaces only work between the interfaces as described. To mitigate this, we recommend using the find-and-replace functionality of text editors like Microsoft Word to ease the translation of syntaxes between other databases. Depending on one’s institutional resources, custom macros can be developed using similar methods.

Results of the method

Whether this method results in exhaustive searches where no important article is missed is difficult to determine, because the number of relevant articles is unknown for any topic. A comparison of several parameters of 73 published reviews that were based on a search developed with this method to 258 reviews that acknowledged information specialists from other Dutch academic hospitals shows that the performance of the searches following our method is comparable to those performed in other institutes but that the time needed to develop the search strategies was much shorter than the time reported for the other reviews [ 9 ].

CONCLUSIONS

With the described method, searchers can gain confidence in their search strategies by finding many relevant words and creating exhaustive search strategies quickly. The approach can be used when performing SR searches or for other purposes such as answering clinical questions, with different expectations of the search’s precision and recall. This method, with practice, provides a stepwise approach that facilitates the search strategy development process from question clarification to final iteration and beyond.

SUPPLEMENTAL FILE

Acknowledgments.

We highly appreciate the work that was done by our former colleague Louis Volkers, who in his twenty years as an information specialist in Erasmus MC laid the basis for our method. We thank Professor Oscar Franco for reviewing earlier drafts of this article.

IMAGES

  1. How To Write A Literature Review

    research literature literature

  2. Sample of Research Literature Review

    research literature literature

  3. 15 Literature Review Examples (2024)

    research literature literature

  4. The Importance of Literature Review in Scientific Research Writing

    research literature literature

  5. How to Write a Stellar Literature Review

    research literature literature

  6. The Literature gap and future research-research process in scientific

    research literature literature

VIDEO

  1. How to Write a Literature Review

  2. Part 2 Writing the Review of Literature

  3. Effective Review of Literature

  4. Research (Literature Review) Groups 9 & 10

  5. Literature review and its process

  6. Review of Literature

COMMENTS

  1. How to Write a Literature Review

    Examples of literature reviews. Step 1 - Search for relevant literature. Step 2 - Evaluate and select sources. Step 3 - Identify themes, debates, and gaps. Step 4 - Outline your literature review's structure. Step 5 - Write your literature review.

  2. Semantic Scholar

    Semantic Scholar is a free, AI-powered research tool for scientific literature, based at the Allen Institute for AI. Learn More. About About Us Meet the Team Publishers Blog (opens in a new tab) AI2 Careers (opens in a new tab) Product Product Overview Semantic Reader Scholar's Hub Beta Program Release Notes. API

  3. Writing a Literature Review

    A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research ...

  4. Literature review as a research methodology: An overview and guidelines

    This is why the literature review as a research method is more relevant than ever. Traditional literature reviews often lack thoroughness and rigor and are conducted ad hoc, rather than following a specific methodology. Therefore, questions can be raised about the quality and trustworthiness of these types of reviews.

  5. 5. The Literature Review

    A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories.A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that ...

  6. Steps in Conducting a Literature Review

    A literature review is an integrated analysis-- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

  7. Reviewing literature for research: Doing it the right way

    Literature search. Fink has defined research literature review as a "systematic, explicit and reproducible method for identifying, evaluating, and synthesizing the existing body of completed and recorded work produced by researchers, scholars and practitioners."[]Review of research literature can be summarized into a seven step process: (i) Selecting research questions/purpose of the ...

  8. Literature Review Research

    Literature Review is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works. Also, we can define a literature review as the ...

  9. Research Guides: Literature Reviews: What is a Literature Review?

    A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the ...

  10. Methodological Approaches to Literature Review

    A literature review is defined as "a critical analysis of a segment of a published body of knowledge through summary, classification, and comparison of prior research studies, reviews of literature, and theoretical articles." (The Writing Center University of Winconsin-Madison 2022) A literature review is an integrated analysis, not just a summary of scholarly work on a specific topic.

  11. Writing a literature review

    Writing a literature review requires a range of skills to gather, sort, evaluate and summarise peer-reviewed published data into a relevant and informative unbiased narrative. Digital access to research papers, academic texts, review articles, reference databases and public data sets are all sources of information that are available to enrich ...

  12. Reviewing the research methods literature: principles and strategies

    The conventional focus of rigorous literature reviews (i.e., review types for which systematic methods have been codified, including the various approaches to quantitative systematic reviews [2-4], and the numerous forms of qualitative and mixed methods literature synthesis [5-10]) is to synthesize empirical research findings from multiple ...

  13. Ten Simple Rules for Writing a Literature Review

    Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications .For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively .Given such mountains of papers, scientists cannot be expected to examine in detail every ...

  14. What is a literature review?

    A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important ...

  15. Literature search for research planning and identification of research

    INTRODUCTION. Literature search is a systematic and well-organised search from the already published data to identify a breadth of good quality references on a specific topic.[] The reasons for conducting literature search are numerous that include drawing information for making evidence-based guidelines, a step in the research method and as part of academic assessment.[]

  16. How to Write a Research Proposal

    A research proposal describes what you will investigate, why it's important, and how you will conduct your research. The format of a research proposal varies between fields, but most proposals will contain at least these elements: Title page; Introduction; Literature review; Research design; Reference list

  17. How to Make a Literature Review in Research (RRL Example)

    Step 2: Research and collect all the scholarly information on the topic that might be pertinent to your study. This includes scholarly articles, books, conventions, conferences, dissertations, and theses—these and any other academic work related to your area of study is called "the literature.".

  18. Reviewing the Research Literature

    The research literature in psychology is enormous—including millions of scholarly articles and books dating to the beginning of the field—and it continues to grow. Although its boundaries are somewhat fuzzy, the research literature definitely does not include self-help and other pop psychology books, dictionary and encyclopedia entries ...

  19. How to write the literature review of your research paper

    The main purpose of the review is to introduce the readers to the need for conducting the said research. A literature review should begin with a thorough literature search using the main keywords in relevant online databases such as Google Scholar, PubMed, etc. Once all the relevant literature has been gathered, it should be organized as ...

  20. Research Methods

    Literary research methods tend to differ from research methods in the hard sciences (such as physics and chemistry). ... For example, we may be studying the relationship between literature and society, between author and text, or the status of a work in the literary canon. We may want to know about a work's form, genre, or thematics. We may ...

  21. Literature Review

    Types of Literature Review are as follows: Narrative literature review: This type of review involves a comprehensive summary and critical analysis of the available literature on a particular topic or research question. It is often used as an introductory section of a research paper. Systematic literature review: This is a rigorous and ...

  22. LibGuides: Literature Reviews: 1. Define your research question

    Identifying a well-defined research question is the first step in the literature review process. For undergraduates, professors will often assign a broad topic for a literature review assignment. You will need to more narrowly define your question before you can begin the research process.

  23. Litmaps

    Our newly released course, Mastering Literature Review with Litmaps, allows instructors to seamlessly bring Litmaps into the classroom to teach fundamental literature review and research concepts. Instructors can seamlessly incorporate these literature review lessons and materials into their own curriculum. Click below to access the course.

  24. Frontiers

    Research methodology: Literature analysis and logical analysis. This article is a literature review, so two main approaches have been adopted: documentary analysis and logical analysis. In terms of literature analysis, a large amount of literature has been consulted in writing this article, and as there is a large body of literature relating to ...

  25. Research Guides: Literature Reviews: Concept Mapping

    A concept map or mind map is a visual representation of knowledge that illustrates relationships between concepts or ideas. It is a tool for organizing and representing information in a hierarchical and interconnected manner. At its core, a concept map consists of nodes, which represent individual concepts or ideas, and links, which depict the relationships between these concepts.

  26. Research Guides: Grey Literature: Sources of Grey Literature

    Start your grey literature search with this guide containing resources and strategies for discovering research produced outside traditional commercial or academic publishing channels. ... We update research reports on our platform each month as they become available through contributing institutes.Research reports provide current analysis on ...

  27. AI Literature Review Generator

    Generate a comprehensive literature review based on a specific research topic. HyperWrite's AI Literature Review Generator is a revolutionary tool that automates the process of creating a comprehensive literature review. Powered by the most advanced AI models, this tool can search and analyze scholarly articles, books, and other resources to identify key themes, methodologies, findings, and ...

  28. Research Guides: Psychology Literature Searching: Information Literacy

    Published research articles can be wrong in their conclusions without intent. Because of the peer review process, disinformation is less common in published academic scholarship. But it still does happen with falsified research and can be hard to detect. The site Retraction Watch keeps track of sources that have been identified as problematic ...

  29. A systematic approach to searching: an efficient and complete method to

    A systematic approach to searching an efficient and complete method to develop literature searches is a crucial skill for researchers and practitioners. This article presents a comprehensive and practical guide to design and conduct effective searches in various databases, using techniques such as truncation, Boolean operators, and PICOT framework.