peer group literature review

Disclaimer » Advertising

• HealthyChildren.org

• Previous Article
• Next Article

What is the Purpose of Peer Review?

What makes a good peer reviewer, how do you decide whether to review a paper, how do you complete a peer review, limitations of peer review, conclusions, research methods: how to perform an effective peer review.

• Split-Screen
• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• CME Quiz Close Quiz
• Open the PDF for in another window
• Get Permissions
• Cite Icon Cite
• Search Site

Elise Peterson Lu , Brett G. Fischer , Melissa A. Plesac , Andrew P.J. Olson; Research Methods: How to Perform an Effective Peer Review. Hosp Pediatr November 2022; 12 (11): e409–e413. https://doi.org/10.1542/hpeds.2022-006764

Download citation file:

• Ris (Zotero)
• Reference Manager

Scientific peer review has existed for centuries and is a cornerstone of the scientific publication process. Because the number of scientific publications has rapidly increased over the past decades, so has the number of peer reviews and peer reviewers. In this paper, drawing on the relevant medical literature and our collective experience as peer reviewers, we provide a user guide to the peer review process, including discussion of the purpose and limitations of peer review, the qualities of a good peer reviewer, and a step-by-step process of how to conduct an effective peer review.

Peer review has been a part of scientific publications since 1665, when the Philosophical Transactions of the Royal Society became the first publication to formalize a system of expert review. 1 , 2   It became an institutionalized part of science in the latter half of the 20 th century and is now the standard in scientific research publications. 3   In 2012, there were more than 28 000 scholarly peer-reviewed journals and more than 3 million peer reviewed articles are now published annually. 3 , 4   However, even with this volume, most peer reviewers learn to review “on the (unpaid) job” and no standard training system exists to ensure quality and consistency. 5   Expectations and format vary between journals and most, but not all, provide basic instructions for reviewers. In this paper, we provide a general introduction to the peer review process and identify common strategies for success as well as pitfalls to avoid.

Modern peer review serves 2 primary purposes: (1) as “a screen before the diffusion of new knowledge” 6   and (2) as a method to improve the quality of published work. 1 , 5  

As screeners, peer reviewers evaluate the quality, validity, relevance, and significance of research before publication to maintain the credibility of the publications they serve and their fields of study. 1 , 2 , 7   Although peer reviewers are not the final decision makers on publication (that role belongs to the editor), their recommendations affect editorial decisions and thoughtful comments influence an article’s fate. 6 , 8  

As advisors and evaluators of manuscripts, reviewers have an opportunity and responsibility to give authors an outside expert’s perspective on their work. 9   They provide feedback that can improve methodology, enhance rigor, improve clarity, and redefine the scope of articles. 5 , 8 , 10   This often happens even if a paper is not ultimately accepted at the reviewer’s journal because peer reviewers’ comments are incorporated into revised drafts that are submitted to another journal. In a 2019 survey of authors, reviewers, and editors, 83% said that peer review helps science communication and 90% of authors reported that peer review improved their last paper. 11  

Expertise: Peer reviewers should be up to date with current literature, practice guidelines, and methodology within their subject area. However, academic rank and seniority do not define expertise and are not actually correlated with performance in peer review. 13  

Professionalism: Reviewers should be reliable and objective, aware of their own biases, and respectful of the confidentiality of the peer review process.

Critical skill : Reviewers should be organized, thorough, and detailed in their critique with the goal of improving the manuscript under their review, regardless of disposition. They should provide constructive comments that are specific and addressable, referencing literature when possible. A peer reviewer should leave a paper better than he or she found it.

Is the manuscript within your area of expertise? Generally, if you are asked to review a paper, it is because an editor felt that you were a qualified expert. In a 2019 survey, 74% of requested reviews were within the reviewer’s area of expertise. 11   This, of course, does not mean that you must be widely published in the area, only that you have enough expertise and comfort with the topic to critique and add to the paper.

Do you have any biases that may affect your review? Are there elements of the methodology, content area, or theory with which you disagree? Some disagreements between authors and reviewers are common, expected, and even helpful. However, if a reviewer fundamentally disagrees with an author’s premise such that he or she cannot be constructive, the review invitation should be declined.

Do you have the time? The average review for a clinical journal takes 5 to 6 hours, though many take longer depending on the complexity of the research and the experience of the reviewer. 1 , 14   Journals vary on the requested timeline for return of reviews, though it is usually 1 to 4 weeks. Peer review is often the longest part of the publication process and delays contribute to slower dissemination of important work and decreased author satisfaction. 15   Be mindful of your schedule and only accept a review invitation if you can reasonably return the review in the requested time.

Once you have determined that you are the right person and decided to take on the review, reply to the inviting e-mail or click the associated link to accept (or decline) the invitation. Journal editors invite a limited number of reviewers at a time and wait for responses before inviting others. A common complaint among journal editors surveyed was that reviewers would often take days to weeks to respond to requests, or not respond at all, making it difficult to find appropriate reviewers and prolonging an already long process. 5  

Now that you have decided to take on the review, it is best of have a systematic way of both evaluating the manuscript and writing the review. Various suggestions exist in the literature, but we will describe our standard procedure for review, incorporating specific do’s and don’ts summarized in Table 1 .

Dos and Don’ts of Peer Review

First, read the manuscript once without making notes or forming opinions to get a sense of the paper as whole. Assess the overall tone and flow and define what the authors identify as the main point of their work. Does the work overall make sense? Do the authors tell the story effectively?

Next, read the manuscript again with an eye toward review, taking notes and formulating thoughts on strengths and weaknesses. Consider the methodology and identify the specific type of research described. Refer to the corresponding reporting guideline if applicable (CONSORT for randomized control trials, STROBE for observational studies, PRISMA for systematic reviews). Reporting guidelines often include a checklist, flow diagram, or structured text giving a minimum list of information needed in a manuscript based on the type of research done. 16   This allows the reviewer to formulate a more nuanced and specific assessment of the manuscript.

Next, review the main findings, the significance of the work, and what contribution it makes to the field. Examine the presentation and flow of the manuscript but do not copy edit the text. At this point, you should start to write your review. Some journals provide a format for their reviews, but often it is up to the reviewer. In surveys of journal editors and reviewers, a review organized by manuscript section was the most favored, 5 , 6   so that is what we will describe here.

As you write your review, consider starting with a brief summary of the work that identifies the main topic, explains the basic approach, and describes the findings and conclusions. 12 , 17   Though not universally included in all reviews, we have found this step to be helpful in ensuring that the work is conveyed clearly enough for the reviewer to summarize it. Include brief notes on the significance of the work and what it adds to current knowledge. Critique the presentation of the work: is it clearly written? Is its length appropriate? List any major concerns with the work overall, such as major methodological flaws or inaccurate conclusions that should disqualify it from publication, though do not comment directly on disposition. Then perform your review by section:

Abstract : Is it consistent with the rest of the paper? Does it adequately describe the major points?

Introduction : This section should provide adequate background to explain the need for the study. Generally, classic or highly relevant studies should be cited, but citations do not have to be exhaustive. The research question and hypothesis should be clearly stated.

Methods: Evaluate both the methods themselves and the way in which they are explained. Does the methodology used meet the needs of the questions proposed? Is there sufficient detail to explain what the authors did and, if not, what needs to be added? For clinical research, examine the inclusion/exclusion criteria, control populations, and possible sources of bias. Reporting guidelines can be particularly helpful in determining the appropriateness of the methods and how they are reported.

Some journals will expect an evaluation of the statistics used, whereas others will have a separate statistician evaluate, and the reviewers are generally not expected to have an exhaustive knowledge of statistical methods. Clarify expectations if needed and, if you do not feel qualified to evaluate the statistics, make this clear in your review.

Results: Evaluate the presentation of the results. Is information given in sufficient detail to assess credibility? Are the results consistent with the methodology reported? Are the figures and tables consistent with the text, easy to interpret, and relevant to the work? Make note of data that could be better detailed in figures or tables, rather than included in the text. Make note of inappropriate interpretation in the results section (this should be in discussion) or rehashing of methods.

Discussion: Evaluate the authors’ interpretation of their results, how they address limitations, and the implications of their work. How does the work contribute to the field, and do the authors adequately describe those contributions? Make note of overinterpretation or conclusions not supported by the data.

The length of your review often correlates with your opinion of the quality of the work. If an article has major flaws that you think preclude publication, write a brief review that focuses on the big picture. Articles that may not be accepted but still represent quality work merit longer reviews aimed at helping the author improve the work for resubmission elsewhere.

Generally, do not include your recommendation on disposition in the body of the review itself. Acceptance or rejection is ultimately determined by the editor and including your recommendation in your comments to the authors can be confusing. A journal editor’s decision on acceptance or rejection may depend on more factors than just the quality of the work, including the subject area, journal priorities, other contemporaneous submissions, and page constraints.

Many submission sites include a separate question asking whether to accept, accept with major revision, or reject. If this specific format is not included, then add your recommendation in the “confidential notes to the editor.” Your recommendation should be consistent with the content of your review: don’t give a glowing review but recommend rejection or harshly criticize a manuscript but recommend publication. Last, regardless of your ultimate recommendation on disposition, it is imperative to use respectful and professional language and tone in your written review.

Although peer review is often described as the “gatekeeper” of science and characterized as a quality control measure, peer review is not ideally designed to detect fundamental errors, plagiarism, or fraud. In multiple studies, peer reviewers detected only 20% to 33% of intentionally inserted errors in scientific manuscripts. 18 , 19   Plagiarism similarly is not detected in peer review, largely because of the huge volume of literature available to plagiarize. Most journals now use computer software to identify plagiarism before a manuscript goes to peer review. Finally, outright fraud often goes undetected in peer review. Reviewers start from a position of respect for the authors and trust the data they are given barring obvious inconsistencies. Ultimately, reviewers are “gatekeepers, not detectives.” 7  

Peer review is also limited by bias. Even with the best of intentions, reviewers bring biases including but not limited to prestige bias, affiliation bias, nationality bias, language bias, gender bias, content bias, confirmation bias, bias against interdisciplinary research, publication bias, conservatism, and bias of conflict of interest. 3 , 4 , 6   For example, peer reviewers score methodology higher and are more likely to recommend publication when prestigious author names or institutions are visible. 20   Although bias can be mitigated both by the reviewer and by the journal, it cannot be eliminated. Reviewers should be mindful of their own biases while performing reviews and work to actively mitigate them. For example, if English language editing is necessary, state this with specific examples rather than suggesting the authors seek editing by a “native English speaker.”

Peer review is an essential, though imperfect, part of the forward movement of science. Peer review can function as both a gatekeeper to protect the published record of science and a mechanism to improve research at the level of individual manuscripts. Here, we have described our strategy, summarized in Table 2 , for performing a thorough peer review, with a focus on organization, objectivity, and constructiveness. By using a systematized strategy to evaluate manuscripts and an organized format for writing reviews, you can provide a relatively objective perspective in editorial decision-making. By providing specific and constructive feedback to authors, you contribute to the quality of the published literature.

Take-home Points

FUNDING: No external funding.

CONFLICT OF INTEREST DISCLOSURES: The authors have indicated they have no potential conflicts of interest to disclose.

Dr Lu performed the literature review and wrote the manuscript. Dr Fischer assisted in the literature review and reviewed and edited the manuscript. Dr Plesac provided background information on the process of peer review, reviewed and edited the manuscript, and completed revisions. Dr Olson provided background information and practical advice, critically reviewed and revised the manuscript, and approved the final manuscript.

Advertising Disclaimer »

Citing articles via

Email alerts.

Affiliations

• Editorial Board
• Editorial Policies
• Pediatrics On Call
• Online ISSN 2154-1671
• Print ISSN 2154-1663
• Pediatrics Open Science
• Hospital Pediatrics
• Pediatrics in Review
• AAP Grand Rounds
• Latest News
• Pediatric Care Online
• Red Book Online
• Pediatric Patient Education
• AAP Toolkits
• AAP Pediatric Coding Newsletter

First 1,000 Days Knowledge Center

Institutions/librarians, group practices, licensing/permissions, integrations, advertising.

• Privacy Statement | Accessibility Statement | Terms of Use | Support Center | Contact Us
• © Copyright American Academy of Pediatrics

This Feature Is Available To Subscribers Only

Sign In or Create an Account

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Peer Review? | Types & Examples

What Is Peer Review? | Types & Examples

Published on December 17, 2021 by Tegan George . Revised on June 22, 2023.

Peer review, sometimes referred to as refereeing , is the process of evaluating submissions to an academic journal. Using strict criteria, a panel of reviewers in the same subject area decides whether to accept each submission for publication.

Peer-reviewed articles are considered a highly credible source due to the stringent process they go through before publication.

There are various types of peer review. The main difference between them is to what extent the authors, reviewers, and editors know each other’s identities. The most common types are:

• Single-blind review
• Double-blind review
• Triple-blind review

Collaborative review

Open review.

Relatedly, peer assessment is a process where your peers provide you with feedback on something you’ve written, based on a set of criteria or benchmarks from an instructor. They then give constructive feedback, compliments, or guidance to help you improve your draft.

Table of contents

What is the purpose of peer review, types of peer review, the peer review process, providing feedback to your peers, peer review example, advantages of peer review, criticisms of peer review, other interesting articles, frequently asked questions about peer reviews.

Many academic fields use peer review, largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the manuscript. For this reason, academic journals are among the most credible sources you can refer to.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure.

Peer assessment is often used in the classroom as a pedagogical tool. Both receiving feedback and providing it are thought to enhance the learning process, helping students think critically and collaboratively.

Prevent plagiarism. Run a free check.

Depending on the journal, there are several types of peer review.

Single-blind peer review

The most common type of peer review is single-blind (or single anonymized) review . Here, the names of the reviewers are not known by the author.

While this gives the reviewers the ability to give feedback without the possibility of interference from the author, there has been substantial criticism of this method in the last few years. Many argue that single-blind reviewing can lead to poaching or intellectual theft or that anonymized comments cause reviewers to be too harsh.

Double-blind peer review

In double-blind (or double anonymized) review , both the author and the reviewers are anonymous.

Arguments for double-blind review highlight that this mitigates any risk of prejudice on the side of the reviewer, while protecting the nature of the process. In theory, it also leads to manuscripts being published on merit rather than on the reputation of the author.

Triple-blind peer review

While triple-blind (or triple anonymized) review —where the identities of the author, reviewers, and editors are all anonymized—does exist, it is difficult to carry out in practice.

Proponents of adopting triple-blind review for journal submissions argue that it minimizes potential conflicts of interest and biases. However, ensuring anonymity is logistically challenging, and current editing software is not always able to fully anonymize everyone involved in the process.

In collaborative review , authors and reviewers interact with each other directly throughout the process. However, the identity of the reviewer is not known to the author. This gives all parties the opportunity to resolve any inconsistencies or contradictions in real time, and provides them a rich forum for discussion. It can mitigate the need for multiple rounds of editing and minimize back-and-forth.

Collaborative review can be time- and resource-intensive for the journal, however. For these collaborations to occur, there has to be a set system in place, often a technological platform, with staff monitoring and fixing any bugs or glitches.

Lastly, in open review , all parties know each other’s identities throughout the process. Often, open review can also include feedback from a larger audience, such as an online forum, or reviewer feedback included as part of the final published product.

While many argue that greater transparency prevents plagiarism or unnecessary harshness, there is also concern about the quality of future scholarship if reviewers feel they have to censor their comments.

In general, the peer review process includes the following steps:

• First, the author submits the manuscript to the editor.
• Reject the manuscript and send it back to the author, or
• Send it onward to the selected peer reviewer(s)
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made.
• Lastly, the edited manuscript is sent back to the author. They input the edits and resubmit it to the editor for publication.

In an effort to be transparent, many journals are now disclosing who reviewed each article in the published product. There are also increasing opportunities for collaboration and feedback, with some journals allowing open communication between reviewers and authors.

It can seem daunting at first to conduct a peer review or peer assessment. If you’re not sure where to start, there are several best practices you can use.

Summarize the argument in your own words

Summarizing the main argument helps the author see how their argument is interpreted by readers, and gives you a jumping-off point for providing feedback. If you’re having trouble doing this, it’s a sign that the argument needs to be clearer, more concise, or worded differently.

If the author sees that you’ve interpreted their argument differently than they intended, they have an opportunity to address any misunderstandings when they get the manuscript back.

Separate your feedback into major and minor issues

It can be challenging to keep feedback organized. One strategy is to start out with any major issues and then flow into the more minor points. It’s often helpful to keep your feedback in a numbered list, so the author has concrete points to refer back to.

Major issues typically consist of any problems with the style, flow, or key points of the manuscript. Minor issues include spelling errors, citation errors, or other smaller, easy-to-apply feedback.

Tip: Try not to focus too much on the minor issues. If the manuscript has a lot of typos, consider making a note that the author should address spelling and grammar issues, rather than going through and fixing each one.

The best feedback you can provide is anything that helps them strengthen their argument or resolve major stylistic issues.

Give the type of feedback that you would like to receive

No one likes being criticized, and it can be difficult to give honest feedback without sounding overly harsh or critical. One strategy you can use here is the “compliment sandwich,” where you “sandwich” your constructive criticism between two compliments.

Be sure you are giving concrete, actionable feedback that will help the author submit a successful final draft. While you shouldn’t tell them exactly what they should do, your feedback should help them resolve any issues they may have overlooked.

As a rule of thumb, your feedback should be:

• Easy to understand
• Constructive

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Below is a brief annotated research example. You can view examples of peer feedback by hovering over the highlighted sections.

Influence of phone use on sleep

Studies show that teens from the US are getting less sleep than they were a decade ago (Johnson, 2019) . On average, teens only slept for 6 hours a night in 2021, compared to 8 hours a night in 2011. Johnson mentions several potential causes, such as increased anxiety, changed diets, and increased phone use.

The current study focuses on the effect phone use before bedtime has on the number of hours of sleep teens are getting.

For this study, a sample of 300 teens was recruited using social media, such as Facebook, Instagram, and Snapchat. The first week, all teens were allowed to use their phone the way they normally would, in order to obtain a baseline.

The sample was then divided into 3 groups:

• Group 1 was not allowed to use their phone before bedtime.
• Group 2 used their phone for 1 hour before bedtime.
• Group 3 used their phone for 3 hours before bedtime.

All participants were asked to go to sleep around 10 p.m. to control for variation in bedtime . In the morning, their Fitbit showed the number of hours they’d slept. They kept track of these numbers themselves for 1 week.

Two independent t tests were used in order to compare Group 1 and Group 2, and Group 1 and Group 3. The first t test showed no significant difference ( p > .05) between the number of hours for Group 1 ( M = 7.8, SD = 0.6) and Group 2 ( M = 7.0, SD = 0.8). The second t test showed a significant difference ( p < .01) between the average difference for Group 1 ( M = 7.8, SD = 0.6) and Group 3 ( M = 6.1, SD = 1.5).

This shows that teens sleep fewer hours a night if they use their phone for over an hour before bedtime, compared to teens who use their phone for 0 to 1 hours.

Peer review is an established and hallowed process in academia, dating back hundreds of years. It provides various fields of study with metrics, expectations, and guidance to ensure published work is consistent with predetermined standards.

• Protects the quality of published research

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. Any content that raises red flags for reviewers can be closely examined in the review stage, preventing plagiarized or duplicated research from being published.

• Gives you access to feedback from experts in your field

Peer review represents an excellent opportunity to get feedback from renowned experts in your field and to improve your writing through their feedback and guidance. Experts with knowledge about your subject matter can give you feedback on both style and content, and they may also suggest avenues for further research that you hadn’t yet considered.

• Helps you identify any weaknesses in your argument

Peer review acts as a first defense, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process. This way, you’ll end up with a more robust, more cohesive article.

While peer review is a widely accepted metric for credibility, it’s not without its drawbacks.

• Reviewer bias

The more transparent double-blind system is not yet very common, which can lead to bias in reviewing. A common criticism is that an excellent paper by a new researcher may be declined, while an objectively lower-quality submission by an established researcher would be accepted.

• Delays in publication

The thoroughness of the peer review process can lead to significant delays in publishing time. Research that was current at the time of submission may not be as current by the time it’s published. There is also high risk of publication bias , where journals are more likely to publish studies with positive findings than studies with negative findings.

• Risk of human error

By its very nature, peer review carries a risk of human error. In particular, falsification often cannot be detected, given that reviewers would have to replicate entire experiments to ensure the validity of results.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Measures of central tendency
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Thematic analysis
• Discourse analysis
• Cohort study
• Ethnography

Research bias

• Implicit bias
• Cognitive bias
• Conformity bias
• Hawthorne effect
• Availability heuristic
• Attrition bias
• Social desirability bias

Peer review is a process of evaluating submissions to an academic journal. Utilizing rigorous criteria, a panel of reviewers in the same subject area decide whether to accept each submission for publication. For this reason, academic journals are often considered among the most credible sources you can use in a research project– provided that the journal itself is trustworthy and well-regarded.

In general, the peer review process follows the following steps: 

• Reject the manuscript and send it back to author, or 
• Send it onward to the selected peer reviewer(s) 
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made. 
• Lastly, the edited manuscript is sent back to the author. They input the edits, and resubmit it to the editor for publication.

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. It also represents an excellent opportunity to get feedback from renowned experts in your field. It acts as a first defense, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process.

Peer-reviewed articles are considered a highly credible source due to this stringent process they go through before publication.

Many academic fields use peer review , largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the published manuscript.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure. 

A credible source should pass the CRAAP test  and follow these guidelines:

• The information should be up to date and current.
• The author and publication should be a trusted authority on the subject you are researching.
• The sources the author cited should be easy to find, clear, and unbiased.
• For a web source, the URL and layout should signify that it is trustworthy.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). What Is Peer Review? | Types & Examples. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/methodology/peer-review/

Is this article helpful?

Tegan George

Other students also liked, what are credible sources & how to spot them | examples, ethical considerations in research | types & examples, applying the craap test & evaluating sources, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• Research Guides

Literature Review: A Self-Guided Tutorial

Peer review.

• Literature Reviews: A Recap
• Reading the Literature
• Using Concept Maps
• Developing Research Questions
• Considering Strong Opinions
• 2. Review discipline styles
• Super Searching
• Finding the Full Text
• Citation Searching This link opens in a new window
• When to stop searching
• Citation Management
• Annotating Articles Tip
• 5. Critically analyze and evaluate
• How to Review the Literature
• Using a Synthesis Matrix
• 7. Write literature review

As part of the scholarly publishing process, authors' manuscripts often go through peer review before they are published. Watch the video below to learn about the peer review process. As you watch the video, consider these questions:

• What is the purpose of peer review?
• What are the advantages of this process?
• What are the limitations?

Credit: CSU Dominguez Hills ( CC BY-NC 4.0 ) Run Time: 2:58

• << Previous: Literature Reviews: A Recap
• Next: Reading the Literature >>
• Last Updated: Feb 22, 2024 10:53 AM
• URL: https://libguides.williams.edu/literature-review

• Open access
• Published: 23 August 2022

Prognostic risk factors for moderate-to-severe exacerbations in patients with chronic obstructive pulmonary disease: a systematic literature review

• John R. Hurst 1 ,
• MeiLan K. Han 2 ,
• Barinder Singh 3 ,
• Sakshi Sharma 4 ,
• Gagandeep Kaur 3 ,
• Enrico de Nigris 5 ,
• Ulf Holmgren 6 &
• Mohd Kashif Siddiqui 3  

Respiratory Research volume  23 , Article number:  213 ( 2022 ) Cite this article

6342 Accesses

20 Citations

33 Altmetric

Metrics details

Chronic obstructive pulmonary disease (COPD) is a leading cause of morbidity and mortality worldwide. COPD exacerbations are associated with a worsening of lung function, increased disease burden, and mortality, and, therefore, preventing their occurrence is an important goal of COPD management. This review was conducted to identify the evidence base regarding risk factors and predictors of moderate-to-severe exacerbations in patients with COPD.

A literature review was performed in Embase, MEDLINE, MEDLINE In-Process, and the Cochrane Central Register of Controlled Trials (CENTRAL). Searches were conducted from January 2015 to July 2019. Eligible publications were peer-reviewed journal articles, published in English, that reported risk factors or predictors for the occurrence of moderate-to-severe exacerbations in adults age ≥ 40 years with a diagnosis of COPD.

The literature review identified 5112 references, of which 113 publications (reporting results for 76 studies) met the eligibility criteria and were included in the review. Among the 76 studies included, 61 were observational and 15 were randomized controlled clinical trials. Exacerbation history was the strongest predictor of future exacerbations, with 34 studies reporting a significant association between history of exacerbations and risk of future moderate or severe exacerbations. Other significant risk factors identified in multiple studies included disease severity or bronchodilator reversibility (39 studies), comorbidities (34 studies), higher symptom burden (17 studies), and higher blood eosinophil count (16 studies).

Conclusions

This systematic literature review identified several demographic and clinical characteristics that predict the future risk of COPD exacerbations. Prior exacerbation history was confirmed as the most important predictor of future exacerbations. These prognostic factors may help clinicians identify patients at high risk of exacerbations, which are a major driver of the global burden of COPD, including morbidity and mortality.

Chronic obstructive pulmonary disease (COPD) is the third leading cause of death worldwide [ 1 ]. Based upon disability-adjusted life-years, COPD ranked sixth out of 369 causes of global disease burden in 2019 [ 2 ]. COPD exacerbations are associated with a worsening of lung function, and increased disease burden and mortality (of those patients hospitalized for the first time with an exacerbation, > 20% die within 1 year of being discharged) [ 3 ]. Furthermore, patients with COPD consider exacerbations or hospitalization due to exacerbations to be the most important disease outcome, having a large impact on their lives [ 4 ]. Therefore, reducing the future risk of COPD exacerbations is a key goal of COPD management [ 5 ].

Being able to predict the level of risk for each patient allows clinicians to adapt treatment and patients to adjust their lifestyle (e.g., through a smoking cessation program) to prevent exacerbations [ 3 ]. As such, identifying high-risk patients using measurable risk factors and predictors that correlate with exacerbations is critical to reduce the burden of disease and prevent a cycle of decline encompassing irreversible lung damage, worsening quality of life (QoL), increasing disease burden, high healthcare costs, and early death.

Prior history of exacerbations is generally thought to be the best predictor of future exacerbations; however, there is a growing body of evidence suggesting other demographic and clinical characteristics, including symptom burden, airflow obstruction, comorbidities, and inflammatory biomarkers, also influence risk [ 6 , 7 , 8 , 9 ]. For example, in the prospective ECLIPSE observational study, the likelihood of patients experiencing an exacerbation within 1 year of follow-up increased significantly depending upon several factors, including prior exacerbation history, forced expiratory volume in 1 s (FEV 1 ), St. George’s Respiratory Questionnaire (SGRQ) score, gastroesophageal reflux, and white blood cell count [ 9 ].

Many studies have assessed predictors of COPD exacerbations across a variety of countries and patient populations. This systematic literature review (SLR) was conducted to identify and compile the evidence base regarding risk factors and predictors of moderate-to-severe exacerbations in patients with COPD.

• Systematic literature review

A comprehensive search strategy was designed to identify English-language studies published in peer-reviewed journals providing data on risk factors or predictors of moderate or severe exacerbations in adults aged ≥ 40 years with a diagnosis of COPD (sample size ≥ 100). The protocol is summarized in Table 1 and the search strategy is listed in Additional file 1 : Table S1. Key biomedical electronic literature databases were searched from January 2015 until July 2019. Other sources were identified via bibliographic searching of relevant systematic reviews.

Study selection process

Implementation and reporting followed the recommendations and standards of the Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) statement [ 10 ]. An independent reviewer conducted the first screening based on titles and abstracts, and a second reviewer performed a quality check of the excluded evidence. A single independent reviewer also conducted the second screening based on full-text articles, with a quality check of excluded evidence performed by a second reviewer. Likewise, data tables of the included studies were generated by one reviewer, and another reviewer performed a quality check of extracted data. Where more than one publication was identified describing a single study or trial, data were compiled into a single entry in the data-extraction table to avoid double counting of patients and studies. One publication was designated as the ‘primary publication’ for the purposes of the SLR, based on the following criteria: most recently published evidence and/or the article that presented the majority of data (e.g., journal articles were preferred over conference abstracts; articles that reported results for the full population were preferred over later articles providing results of subpopulations). Other publications reporting results from the same study were designated as ‘linked publications’; any additional data in the linked publications that were not included in the primary publication were captured in the SLR. Conference abstracts were excluded from the SLR unless they were a ‘linked publication.’

Included studies

A total of 5112 references (Fig.  1 ) were identified from the database searches. In total, 76 studies from 113 publications were included in the review. Primary publications and ‘linked publications’ for each study are detailed in Additional file 1 : Table S2, and study characteristics are shown in Additional file 1 : Table S3. The studies included clinical trials, registry studies, cross-sectional studies, cohort studies, database studies, and case–control studies. All 76 included studies were published in peer-reviewed journals. Regarding study design, 61 of the studies were observational (34 retrospective observational studies, 19 prospective observational studies, four cross-sectional studies, two studies with both retrospective and prospective cohort data, one case–control study, and one with cross-sectional and longitudinal data) and 15 were randomized controlled clinical trials.

PRISMA flow diagram of studies through the systematic review process. CA conference abstract, CENTRAL Cochrane Central Register of Controlled Trials, PRISMA  Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Of the 76 studies, 16 were conducted in North America (13 studies in the USA, two in Canada, and one in Mexico); 26 were conducted in Europe (seven studies in Spain, four in the UK, three in Denmark, two studies each in Bulgaria, the Netherlands, and Switzerland, and one study each in Sweden, Serbia, Portugal, Greece, Germany, and France) and 17 were conducted in Asia (six studies in South Korea, four in China, three in Taiwan, two in Japan, and one study each in Singapore and Israel). One study each was conducted in Turkey and Australia. Fifteen studies were conducted across multiple countries.

The majority of the studies (n = 54) were conducted in a multicenter setting, while 22 studies were conducted in a single-center setting. The sample size among the included studies varied from 118 to 339,389 patients.

Patient characteristics

A total of 75 studies reported patient characteristics (Additional file 1 : Table S4). The mean age was reported in 65 studies and ranged from 58.0 to 75.2 years. The proportion of male patients ranged from 39.7 to 97.6%. The majority of included studies (85.3%) had a higher proportion of males than females.

Exacerbation history (as defined per each study) was reported in 18 of 76 included studies. The proportion of patients with no prior exacerbation was reported in ten studies (range, 0.1–79.5% of patients), one or fewer prior exacerbation in ten studies (range, 46–100%), one or more prior exacerbation in eight studies (range, 18.4–100%), and two or more prior exacerbations in 12 studies (range, 6.1–55.0%).

Prognostic factors of exacerbations

A summary of the risk factors and predictors reported across the included studies is provided in Tables 2 and 3 . The overall findings of the SLR are summarized in Figs. 2 and 3 .

Risk factors for moderate-to-severe exacerbations in patients with COPD. Factors with > 30 supporting studies shown as large circles; factors with ≤ 30 supporting studies shown as small circles and should be interpreted cautiously. BDR bronchodilator reversibility, BMI body mass index, COPD chronic obstructive pulmonary disease, EOS eosinophil, QoL quality of life

Summary of risk factors for exacerbation events. a Treatment impact studies removed. BDR bronchodilator reversibility, BMI body mass index, COPD chronic obstructive pulmonary disease, EOS eosinophil, QoL quality of life

Exacerbation history within the past 12 months was the strongest predictor of future exacerbations. Across the studies assessing this predictor, 34 out of 35 studies (97.1%) reported a significant association between history of exacerbations and risk of future moderate-to-severe exacerbations (Table 3 ). Specifically, two or more exacerbations in the previous year or at least one hospitalization for COPD in the previous year were identified as reliable predictors of future moderate or severe exacerbations. Even one moderate exacerbation increased the risk of a future exacerbation, with the risk increasing further with each subsequent exacerbation (Fig.  4 ). A severe exacerbation was also found to increase the risk of subsequent exacerbation and hospitalization (Fig.  5 ). Patients experiencing one or more severe exacerbations were more likely to experience further severe exacerbations than moderate exacerbations [ 11 , 12 ]. In contrast, patients with a history of one or more moderate exacerbations were more likely to experience further moderate exacerbations than severe exacerbations [ 11 , 12 ].

Exacerbation history as a risk factor for moderate-to-severe exacerbations. Yun 2018 included two studies; the study from which data were extracted (COPDGene or ECLIPSE) is listed in parentheses. CI confidence interval, ES effect size

Exacerbation history as a risk factor for severe exacerbations. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, ES , effect size

Overall, 35 studies assessed the association of comorbidities with the risk of exacerbation. All studies except one (97.1%) reported a positive association between comorbidities and the occurrence of moderate-to-severe exacerbations (Table 3 ). In addition to the presence of any comorbidity, specific comorbidities that were found to significantly increase the risk of moderate-to-severe exacerbations included anxiety and depression, cardiovascular comorbidities, gastroesophageal reflux disease/dyspepsia, and respiratory comorbidities (Fig.  6 ). Comorbidities that were significant risk factors for severe exacerbations included cardiovascular, musculoskeletal, and respiratory comorbidities, diabetes, and malignancy (Fig.  7 ). Overall, the strongest association between comorbidities and COPD readmissions in the emergency department was with cardiovascular disease. The degree of risk for both moderate-to-severe and severe exacerbations also increased with the number of comorbidities. A Dutch cohort study found that 88% of patients with COPD had at least one comorbidity, with hypertension (35%) and coronary heart disease (19%) being the most prevalent. In this cohort, the comorbidities with the greatest risk of frequent exacerbations were pulmonary cancer (odds ratio [OR] 1.85) and heart failure (OR 1.72) [ 7 ].

Comorbidities as risk factors for moderate-to-severe exacerbations. Yun 2018 included two studies; the study from which data were extracted (COPDGene or ECLIPSE) is listed in parentheses. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, ES effect size, GERD gastroesophageal disease

Comorbidities as risk factors for severe exacerbations. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, CKD , chronic kidney disease, ES effect size

The majority of studies assessing disease severity or bronchodilator reversibility (39/41; 95.1%) indicated a significant positive relation between risk of future exacerbations and greater disease severity, as assessed by greater lung function impairment (in terms of lower FEV 1 , FEV 1 /forced vital capacity ratio, or forced expiratory flow [25–75]/forced vital capacity ratio) or more severe Global Initiative for Chronic Obstructive Lung Disease (GOLD) class A − D, and a positive relationship between risk of future exacerbations and lack of bronchodilator reversibility (Table 3 , Figs. 8 and 9 ).

Disease severity as a risk factor for moderate-to-severe exacerbations. Yun 2018 included two studies; the study from which data were extracted (COPDGene or ECLIPSE) is listed in parentheses. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, ES effect size, FEV 1 f orced expiratory volume in 1 s, FVC , forced vital capacity, GOLD Global Initiative for Obstructive Lung Disease, HR hazard ratio, OR odds ratio

Disease severity and BDR as risk factors for severe exacerbations. ACCP American College of Chest Physicians, ACOS Asthma-COPD overlap syndrome, ATS  American Thoracic Society, BDR bronchodilator reversibility, CI confidence interval, ERS  European Respiratory Society, ES effect size, FEV 1 forced expiratory volume in 1 s, FVC  forced vital capacity, GINA Global Initiative for Asthma, GOLD Global Initiative for Obstructive Lung Disease

Of 21 studies assessing the relationship between blood eosinophil count and exacerbations (Table 3 ), 16 reported estimates for the risk of moderate or severe exacerbations by eosinophil count. A positive association was observed between higher eosinophil count and a higher risk of moderate or severe exacerbations, particularly in patients not treated with an inhaled corticosteroid (ICS); however, five studies reported a significant positive association irrespective of intervention effects. The risk of moderate-to-severe exacerbations was observed to be positively associated with various definitions of higher eosinophil levels (absolute counts: ≥ 200, ≥ 300, ≥ 340, ≥ 400, and ≥ 500 cells/mm 3 ; % of blood eosinophil count: ≥ 2%, ≥ 3%, ≥ 4%, and ≥ 5%). Of note, one study found reduced efficacy of ICS in lowering moderate-to-severe exacerbation rates for current smokers versus former smokers at all eosinophil levels [ 13 ].

Of 12 studies assessing QoL scales, 11 (91.7%) studies reported a significant association between the worsening of QoL scores and the risk of future exacerbations (Table 3 ). Baseline SGRQ [ 14 , 15 ], Center for Epidemiologic Studies Depression Scale (for which increased scores may indicate impaired QoL) [ 16 ], and Clinical COPD Questionnaire [ 17 , 18 ] scores were found to be associated with future risk of moderate and/or severe COPD exacerbations. For symptom scores, six out of eight studies assessing the association between moderate-to-severe or severe exacerbations with COPD Assessment Test (CAT) scores reported a significant and positive relationship. Furthermore, the risk of moderate-to-severe exacerbations was found to be significantly higher in patients with higher CAT scores (≥ 10) [ 15 , 19 , 20 , 21 ], with one study demonstrating that a CAT score of 15 increased predictive ability for exacerbations compared with a score of 10 or more [ 18 ]. Among 15 studies that assessed the association of modified Medical Research Council (mMRC) scores with the risk of moderate-to-severe or severe exacerbation, 11 found that the risk of moderate-to-severe or severe exacerbations was significantly associated with higher mMRC scores (≥ 2) versus lower scores. Furthermore, morning and night symptoms (measured by Clinical COPD Questionnaire) were associated with poor health status and predicted future exacerbations [ 17 ].

Of 36 studies reporting the relationship between smoking status and moderate-to-severe or severe exacerbations, 22 studies (61.1%) reported a significant positive association (Table 3 ). Passive smoking was also significantly associated with an increased risk of severe exacerbations (OR 1.49) [ 20 ]. Of note, three studies reported a significantly lower rate of moderate-to-severe exacerbations in current smokers compared with former smokers [ 22 , 23 , 24 ].

A total of 14 studies assessed the association of body mass index (BMI) with the occurrence of frequent moderate-to-severe exacerbations in patients with COPD. Six out of 14 studies (42.9%) reported a significant negative association between exacerbations and BMI (Table 3 ). The risk of moderate and/or severe COPD exacerbations was highest among underweight patients compared with normal and overweight patients [ 23 , 25 , 26 , 27 , 28 ].

In the 29 studies reporting an association between age and moderate or severe exacerbations, more than half found an association of older age with an increased risk of moderate-to-severe exacerbations (58.6%; Table 3 ). Four of these studies noted a significant increase in the risk of moderate-to-severe or severe exacerbations for every 10-year increase in age [ 25 , 26 , 29 , 30 ]. However, 12 studies reported no significant association between age and moderate-to-severe or severe exacerbation risk.

Sixteen out of 33 studies investigating the impact of sex on exacerbation risk found a significant association (48.5%; Table 3 ). Among these, ten studies reported that female sex was associated with an increased risk of moderate-to-severe exacerbations, while six studies showed a higher exacerbation risk in males compared with females. There was some variation in findings by geographic location and exacerbation severity (Additional file 2 : Figs. S1 and S2). Notably, when assessing the risk of severe exacerbations, more studies found an association with male sex compared with female sex (6/13 studies vs 1/13 studies, respectively).

Both studies evaluating associations between exacerbations and environmental factors reported that colder temperature and exposure to major air pollution (NO 2 , O 3 , CO, and/or particulate matter ≤ 10 μm in diameter) increased hospital admissions due to severe exacerbations and moderate-to-severe exacerbation rates [ 31 , 32 ].

Four studies assessed the association of 6-min walk distance with the occurrence of frequent moderate-to-severe exacerbations (Table 3 ). One study (25.0%) found that shorter 6-min walk distance (representing low physical activity) was significantly associated with a shortened time to severe exacerbation, but the effect size was small (hazard ratio 0.99) [ 33 ].

Five out of six studies assessing the relationship between race or ethnicity and exacerbation risk reported significant associations (Table 3 ). Additionally, one study reported an association between geographic location in the US and exacerbations, with living in the Northeast region being the strongest predictor of severe COPD exacerbations versus living in the Midwest and South regions [ 34 ].

Overall, seven studies assessed the association of biomarkers with risk of future exacerbations (Table 3 ), with the majority identifying significant associations between inflammatory biomarkers and increased exacerbation risk, including higher C-reactive protein levels [ 8 , 35 ], fibrinogen levels [ 8 , 30 ], and white blood cell count [ 8 , 15 , 16 ].

This SLR has identified several demographic and clinical characteristics that predict the future risk of COPD exacerbations. Key factors associated with an increased risk of future moderate-to-severe exacerbations included a history of prior exacerbations, worse disease severity and bronchodilator reversibility, the presence of comorbidities, a higher eosinophil count, and older age (Fig.  2 ). These prognostic factors may help clinicians identify patients at high risk of exacerbations, which are a major driver of the burden of COPD, including morbidity and mortality [ 36 ].

Findings from this review summarize the existing evidence, validating the previously published literature [ 6 , 9 , 23 ] and suggesting that the best predictor of future exacerbations is a history of exacerbations in the prior year [ 8 , 11 , 12 , 13 , 14 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 26 , 29 , 34 , 35 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 ]. In addition, the effect size generally increased with the number of prior exacerbations, with a stronger effect observed with prior severe versus moderate exacerbations. This effect was observed across regions, including in Europe and North America, and in several global studies. This relationship represents a vicious circle, whereby one exacerbation predisposes a patient to experience future exacerbations and leading to an ever-increasing disease burden, and emphasizes the importance of preventing the first exacerbation event through early, proactive exacerbation prevention. The finding that prior exacerbations tended to be associated with future exacerbations of the same severity suggests that the severity of the underlying disease may influence exacerbation severity. However, the validity of the traditional classification of exacerbation severity has recently been challenged [ 61 ], and further work is required to understand relationships with objective assessments of exacerbation severity.

In addition to exacerbation history, disease severity and bronchodilator reversibility were also strong predictors for future exacerbations [ 8 , 14 , 16 , 18 , 19 , 20 , 22 , 23 , 24 , 26 , 28 , 29 , 33 , 37 , 40 , 43 , 44 , 45 , 46 , 48 , 50 , 51 , 52 , 56 , 59 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 ]. The association with disease severity was noted in studies that used GOLD disease stages 1–4 and those that used FEV 1 percent predicted and other lung function assessments as continuous variables. Again, this risk factor is self-perpetuating, as evidence shows that even a single moderate or severe exacerbation may almost double the rate of lung function decline [ 79 ]. Accordingly, disease severity and exacerbation history may be correlated. Margüello et al. concluded that the severity of COPD could be associated with a higher risk of exacerbations, but this effect was partly determined by the exacerbations suffered in the previous year [ 23 ]. It should be noted that FEV 1 is not recommended by GOLD for use as a predictor of exacerbation risk or mortality alone due to insufficient precision when used at the individual patient level [ 5 ].

Another factor that should be considered when assessing individual exacerbation risk is the presence of comorbidities [ 7 , 14 , 16 , 18 , 19 , 20 , 21 , 22 , 24 , 25 , 26 , 27 , 28 , 30 , 33 , 34 , 35 , 40 , 41 , 44 , 45 , 46 , 47 , 48 , 51 , 52 , 53 , 54 , 56 , 58 , 59 , 63 , 64 , 73 , 74 , 76 , 77 , 80 , 81 , 82 , 83 , 84 , 85 ]. Comorbidities are common in COPD, in part due to common risk factors (e.g., age, smoking, lifestyle factors) that also increase the risk of other chronic diseases [ 7 ]. Significant associations were observed between exacerbation risk and comorbidities, such as anxiety and depression, cardiovascular disease, diabetes, and respiratory comorbidities. As with prior exacerbations, the strength of the association increased with the number of comorbidities. Some comorbidities that were found to be associated with COPD exacerbations share a common biological mechanism of systemic inflammation, such as cardiovascular disease, diabetes, and depression [ 86 ]. Furthermore, other respiratory comorbidities, including asthma and bronchiectasis, involve inflammation of the airways [ 87 ]. In these patients, optimal management of comorbidities may reduce the risk of future COPD exacerbations (and improve QoL), although further research is needed to confirm the efficacy of this approach to exacerbation prevention. As cardiovascular conditions, including hypertension and coronary heart disease, are the most common comorbidities in people with COPD [ 7 ], reducing cardiovascular risk may be a key goal in reducing the occurrence of exacerbations. For other comorbidities, the mechanism for the association with exacerbation risk may be related to non-biological factors. For example, in depression, it has been suggested that the mechanism may relate to greater sensitivity to symptom changes or more frequent physician visits [ 88 ].

There is now a growing body of evidence reporting the relationship between blood eosinophil count and exacerbation risk [ 8 , 13 , 14 , 20 , 37 , 48 , 52 , 56 , 59 , 60 , 62 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 ]. Data from many large clinical trials (SUNSET [ 89 ], FLAME [ 96 ], WISDOM [ 98 ], IMPACT [ 13 ], TRISTAN [ 99 ], INSPIRE [ 99 ], KRONOS [ 91 ], TRIBUTE [ 48 ], TRILOGY [ 52 ], TRINITY [ 56 ]) have also shown relationships between treatment, eosinophil count, and exacerbation rates. Evidence shows that eosinophil count, along with other effect modifiers (e.g., exacerbation history), can be used to predict reductions in exacerbations with ICS treatment. Identifying patients most likely to respond to ICS should contribute to personalized medicine approaches to treat COPD. One challenge in drawing a strong conclusion from eosinophil counts is the choice of a cut-off value, with a variety of absolute and percentage values observed to be positively associated with the risk of moderate-to-severe exacerbations. The use of absolute counts may be more practical, as these are not affected by variations in other immune cell numbers; however, there is a lack of consensus on this point [ 100 ].

Across the studies examined, associations between sex and the risk of moderate and/or severe exacerbations were variable [ 14 , 16 , 18 , 20 , 21 , 22 , 23 , 24 , 26 , 27 , 28 , 29 , 37 , 40 , 42 , 44 , 45 , 46 , 47 , 48 , 51 , 52 , 56 , 58 , 59 , 63 , 73 , 74 , 77 , 80 , 83 , 84 , 85 ]. A greater number of studies showed an increased risk of exacerbations in females compared with males. In contrast, some studies failed to detect a relationship, suggesting that country-specific or cultural factors may play a role. A majority of the included studies evaluated more male patients than female patients; to further elucidate the relationship between sex and exacerbations, more studies in female patients are warranted. Over half of the studies that assessed the relationship between age and exacerbation risk found an association between increasing age and increasing risk of moderate-to-severe COPD exacerbations [ 14 , 16 , 18 , 20 , 21 , 22 , 23 , 24 , 26 , 27 , 28 , 29 , 33 , 40 , 42 , 44 , 45 , 47 , 51 , 52 , 54 , 56 , 63 , 73 , 74 , 77 , 80 , 83 , 85 ].

Our findings also suggested that patients with low BMI have greater risk of moderate and/or severe exacerbations. The mechanism underlying this increased risk in underweight patients is poorly understood; however, loss of lean body mass in patients with COPD may be related to ongoing systemic inflammation that impacts skeletal muscle mass [ 101 , 102 , 103 ].

A limitation of this SLR, that may have resulted in some studies with valid results being missed, was the exclusion of non-English-language studies and the limitation by date; however, the search strategy was otherwise broad, resulting in the review of a large number of studies. The majority of studies captured in this SLR were from Europe, North America, and Asia. The findings may therefore be less generalizable to patients in other regions, such as Africa or South America. Given that one study reported an association between geographic location within different regions of the US and exacerbations [ 34 ], it is plausible that risk of exacerbations may be impacted by global location. As no formal meta-analysis was planned, the assessments are based on a qualitative synthesis of studies. A majority of the included studies looked at exposures of certain factors (e.g., history of exacerbations) at baseline; however, some of these factors change over time, calling into question whether a more sophisticated statistical analysis should have been conducted in some cases to consider time-varying covariates. Our results can only inform on associations, not causation, and there are likely bidirectional relationships between many factors and exacerbation risk (e.g., health status). Finally, while our review of the literature captured a large number of prognostic factors, other variables such as genetic factors, lung microbiome composition, and changes in therapy over time have not been widely studied to date, but might also influence exacerbation frequency [ 104 ]. Further research is needed to assess the contribution of these factors to exacerbation risk.

This SLR captured publications up to July 2019. However, further studies have since been published that further support the prognostic factors identified here. For example, recent studies have reported an increased risk of exacerbations in patients with a history of exacerbations [ 105 ], comorbidities [ 106 ], poorer lung function (GOLD stage) [ 105 ], higher symptomatic burden [ 107 ], female sex [ 105 ], and lower BMI [ 106 , 108 ].

In summary, the literature assessing risk factors for moderate-to-severe COPD exacerbations shows that there are associations between several demographic and disease characteristics with COPD exacerbations, potentially allowing clinicians to identify patients most at risk of future exacerbations. Exacerbation history, comorbidities, and disease severity or bronchodilator reversibility were the factors most strongly associated with exacerbation risk, and should be considered in future research efforts to develop prognostic tools to estimate the likelihood of exacerbation occurrence. Importantly, many prognostic factors for exacerbations, such as symptom burden, QoL, and comorbidities, are modifiable with optimal pharmacologic and non-pharmacologic treatments or lifestyle modifications. Overall, the evidence suggests that, taken together, predicting and reducing exacerbation risk is an achievable goal in COPD.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Body mass index

COPD Assessment Test

Chronic obstructive pulmonary disease

Forced expiratory volume in 1 s

Global Initiative for Chronic Obstructive Lung Disease

Inhaled corticosteroid

Modified Medical Research Council

Quality of life

St. George’s Respiratory Questionnaire

World Health Organization. The top 10 causes of death. 2018. https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death . Accessed 22 Jul 2020.

GBD 2019 Diseases and Injuries Collaborators. Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. The Lancet. 2020;396:1204–22.

Article   Google Scholar  

Hurst JR, Skolnik N, Hansen GJ, Anzueto A, Donaldson GC, Dransfield MT, Varghese P. Understanding the impact of chronic obstructive pulmonary disease exacerbations on patient health and quality of life. Eur J Intern Med. 2020;73:1–6.

Article   PubMed   Google Scholar  

Zhang Y, Morgan RL, Alonso-Coello P, Wiercioch W, Bała MM, Jaeschke RR, Styczeń K, Pardo-Hernandez H, Selva A, Ara Begum H, et al. A systematic review of how patients value COPD outcomes. Eur Respir J. 2018;52:1800222.

Global Initiative for Chronic Obstructive Lung Disease. 2022 GOLD Report. Global strategy for the diagnosis, management and prevention of COPD. 2022. https://goldcopd.org/2022-gold-reports-2/ . Accessed 02 Feb 2022.

Müllerová H, Shukla A, Hawkins A, Quint J. Risk factors for acute exacerbations of COPD in a primary care population: a retrospective observational cohort study. BMJ Open. 2014;4: e006171.

Article   PubMed   PubMed Central   Google Scholar  

Westerik JAM, Metting EI, van Boven JFM, Tiersma W, Kocks JWH, Schermer TR. Associations between chronic comorbidity and exacerbation risk in primary care patients with COPD. Respir Res. 2017;18:31.

Vedel-Krogh S, Nielsen SF, Lange P, Vestbo J, Nordestgaard BG. Blood eosinophils and exacerbations in chronic obstructive pulmonary disease. The Copenhagen General Population Study. Am J Respir Crit Care Med. 2016;193:965–74.

Hurst JR, Vestbo J, Anzueto A, Locantore N, Müllerová H, Tal-Singer R, Miller B, Lomas DA, Agusti A, Macnee W, et al. Susceptibility to exacerbation in chronic obstructive pulmonary disease. N Engl J Med. 2010;363:1128–38.

Article   CAS   PubMed   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.

Çolak Y, Afzal S, Marott JL, Nordestgaard BG, Vestbo J, Ingebrigtsen TS, Lange P. Prognosis of COPD depends on severity of exacerbation history: a population-based analysis. Respir Med. 2019;155:141–7.

Rothnie KJ, Müllerová H, Smeeth L, Quint JK. Natural history of chronic obstructive pulmonary disease exacerbations in a general practice-based population with chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2018;198:464–71.

Pascoe S, Barnes N, Brusselle G, Compton C, Criner GJ, Dransfield MT, Halpin DMG, Han MK, Hartley B, Lange P, et al. Blood eosinophils and treatment response with triple and dual combination therapy in chronic obstructive pulmonary disease: analysis of the IMPACT trial. Lancet Respir Med. 2019;7:745–56.

Yun JH, Lamb A, Chase R, Singh D, Parker MM, Saferali A, Vestbo J, Tal-Singer R, Castaldi PJ, Silverman EK, et al. Blood eosinophil count thresholds and exacerbations in patients with chronic obstructive pulmonary disease. J Allergy Clin Immunol. 2018;141:2037-2047.e10.

Yoon HY, Park SY, Lee CH, Byun MK, Na JO, Lee JS, Lee WY, Yoo KH, Jung KS, Lee JH. Prediction of first acute exacerbation using COPD subtypes identified by cluster analysis. Int J Chron Obstruct Pulmon Dis. 2019;14:1389–97.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Yohannes AM, Mulerova H, Lavoie K, Vestbo J, Rennard SI, Wouters E, Hanania NA. The association of depressive symptoms with rates of acute exacerbations in patients with COPD: results from a 3-year longitudinal follow-up of the ECLIPSE cohort. J Am Med Dir Assoc. 2017;18:955-959.e6.

Tsiligianni I, Metting E, van der Molen T, Chavannes N, Kocks J. Morning and night symptoms in primary care COPD patients: a cross-sectional and longitudinal study. An UNLOCK study from the IPCRG. NPJ Prim Care Respir Med. 2016;26:16040.

Jo YS, Yoon HI, Kim DK, Yoo CG, Lee CH. Comparison of COPD Assessment Test and Clinical COPD Questionnaire to predict the risk of exacerbation. Int J Chron Obstruct Pulmon Dis. 2018;13:101–7.

Marçôa R, Rodrigues DM, Dias M, Ladeira I, Vaz AP, Lima R, Guimarães M. Classification of Chronic Obstructive Pulmonary Disease (COPD) according to the new Global Initiative for Chronic Obstructive Lung Disease (GOLD) 2017: comparison with GOLD 2011. COPD. 2018;15:21–6.

Han MK, Quibrera PM, Carretta EE, Barr RG, Bleecker ER, Bowler RP, Cooper CB, Comellas A, Couper DJ, Curtis JL, et al. Frequency of exacerbations in patients with chronic obstructive pulmonary disease: an analysis of the SPIROMICS cohort. Lancet Respir Med. 2017;5:619–26.

Yii ACA, Loh CH, Tiew PY, Xu H, Taha AAM, Koh J, Tan J, Lapperre TS, Anzueto A, Tee AKH. A clinical prediction model for hospitalized COPD exacerbations based on “treatable traits.” Int J Chron Obstruct Pulmon Dis. 2019;14:719–28.

McGarvey L, Lee AJ, Roberts J, Gruffydd-Jones K, McKnight E, Haughney J. Characterisation of the frequent exacerbator phenotype in COPD patients in a large UK primary care population. Respir Med. 2015;109:228–37.

Margüello MS, Garrastazu R, Ruiz-Nuñez M, Helguera JM, Arenal S, Bonnardeux C, León C, Miravitlles M, García-Rivero JL. Independent effect of prior exacerbation frequency and disease severity on the risk of future exacerbations of COPD: a retrospective cohort study. NPJ Prim Care Respir Med. 2016;26:16046.

Engel B, Schindler C, Leuppi JD, Rutishauser J. Predictors of re-exacerbation after an index exacerbation of chronic obstructive pulmonary disease in the REDUCE randomised clinical trial. Swiss Med Wkly. 2017;147: w14439.

PubMed   Google Scholar  

Benson VS, Müllerová H, Vestbo J, Wedzicha JA, Patel A, Hurst JR. Evaluation of COPD longitudinally to identify predictive surrogate endpoints (ECLIPSE) investigators. Associations between gastro-oesophageal reflux, its management and exacerbations of chronic obstructive pulmonary disease. Respir Med. 2015;109:1147–54.

Santibáñez M, Garrastazu R, Ruiz-Nuñez M, Helguera JM, Arenal S, Bonnardeux C, León C, García-Rivero JL. Predictors of hospitalized exacerbations and mortality in chronic obstructive pulmonary disease. PLoS ONE. 2016;11: e0158727.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Jo YS, Kim YH, Lee JY, Kim K, Jung KS, Yoo KH, Rhee CK. Impact of BMI on exacerbation and medical care expenses in subjects with mild to moderate airflow obstruction. Int J Chron Obstruct Pulmon Dis. 2018;13:2261–9.

Alexopoulos EC, Malli F, Mitsiki E, Bania EG, Varounis C, Gourgoulianis KI. Frequency and risk factors of COPD exacerbations and hospitalizations: a nationwide study in Greece (Greek Obstructive Lung Disease Epidemiology and health ecoNomics: GOLDEN study). Int J Chron Obstruct Pulmon Dis. 2015;10:2665–74.

PubMed   PubMed Central   Google Scholar  

Liu D, Peng SH, Zhang J, Bai SH, Liu HX, Qu JM. Prediction of short term re-exacerbation in patients with acute exacerbation of chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis. 2015;10:1265–73.

Müllerová H, Maselli DJ, Locantore N, Vestbo J, Hurst JR, Wedzicha JA, Bakke P, Agusti A, Anzueto A. Hospitalized exacerbations of COPD: risk factors and outcomes in the ECLIPSE cohort. Chest. 2015;147:999–1007.

de Miguel-Díez J, Hernández-Vázquez J, López-de-Andrés A, Álvaro-Meca A, Hernández-Barrera V, Jiménez-García R. Analysis of environmental risk factors for chronic obstructive pulmonary disease exacerbation: a case-crossover study (2004–2013). PLoS ONE. 2019;14: e0217143.

Krachunov II, Kyuchukov NH, Ivanova ZI, Yanev NA, Hristova PA, Borisova ED, Popova TP, Pavlov PS, Nikolova PT, Ivanov YY. Impact of air pollution and outdoor temperature on the rate of chronic obstructive pulmonary disease exacerbations. Folia Med (Plovdiv). 2017;59:423–9.

Article   CAS   Google Scholar  

Baumeler L, Papakonstantinou E, Milenkovic B, Lacoma A, Louis R, Aerts JG, Welte T, Kostikas K, Blasi F, Boersma W, et al. Therapy with proton-pump inhibitors for gastroesophageal reflux disease does not reduce the risk for severe exacerbations in COPD. Respirology. 2016;21:883–90.

Annavarapu S, Goldfarb S, Gelb M, Moretz C, Renda A, Kaila S. Development and validation of a predictive model to identify patients at risk of severe COPD exacerbations using administrative claims data. Int J Chron Obstruct Pulmon Dis. 2018;13:2121–30.

Crisafulli E, Torres A, Huerta A, Méndez R, Guerrero M, Martinez R, Liapikou A, Soler N, Sethi S, Menéndez R. C-reactive protein at discharge, diabetes mellitus and ≥1 hospitalization during previous year predict early readmission in patients with acute exacerbation of chronic obstructive pulmonary disease. COPD. 2015;12:311–20.

Bollmeier SG, Hartmann AP. Management of chronic obstructive pulmonary disease: a review focusing on exacerbations. Am J Health Syst Pharm. 2020;77:259–68.

Bafadhel M, Peterson S, De Blas MA, Calverley PM, Rennard SI, Richter K, Fagerås M. Predictors of exacerbation risk and response to budesonide in patients with chronic obstructive pulmonary disease: a post-hoc analysis of three randomised trials. Lancet Respir Med. 2018;6:117–26.

Calverley PM, Anzueto AR, Dusser D, Mueller A, Metzdorf N, Wise RA. Treatment of exacerbations as a predictor of subsequent outcomes in patients with COPD. Int J Chron Obstruct Pulmon Dis. 2018;13:1297–308.

Calverley PM, Tetzlaff K, Dusser D, Wise RA, Mueller A, Metzdorf N, Anzueto A. Determinants of exacerbation risk in patients with COPD in the TIOSPIR study. Int J Chron Obstruct Pulmon Dis. 2017;12:3391–405.

Eklöf J, Sørensen R, Ingebrigtsen TS, Sivapalan P, Achir I, Boel JB, Bangsborg J, Ostergaard C, Dessau RB, Jensen US, et al. Pseudomonas aeruginosa and risk of death and exacerbations in patients with chronic obstructive pulmonary disease: an observational cohort study of 22 053 patients. Clin Microbiol Infect. 2020;26:227–34.

Estirado C, Ceccato A, Guerrero M, Huerta A, Cilloniz C, Vilaró O, Gabarrús A, Gea J, Crisafulli E, Soler N, Torres A. Microorganisms resistant to conventional antimicrobials in acute exacerbations of chronic obstructive pulmonary disease. Respir Res. 2018;19:119.

Fuhrman C, Moutengou E, Roche N, Delmas MC. Prognostic factors after hospitalization for COPD exacerbation. Rev Mal Respir. 2017;34:1–18.

Krachunov I, Kyuchukov N, Ivanova Z, Yanev NA, Hristova PA, Pavlov P, Glogovska P, Popova T, Ivanov YY. Stability of frequent exacerbator phenotype in patients with chronic obstructive pulmonary disease. Folia Med (Plovdiv). 2018;60:536–45.

Make BJ, Eriksson G, Calverley PM, Jenkins CR, Postma DS, Peterson S, Östlund O, Anzueto A. A score to predict short-term risk of COPD exacerbations (SCOPEX). Int J Chron Obstruct Pulmon Dis. 2015;10:201–9.

Montserrat-Capdevila J, Godoy P, Marsal JR, Barbé F. Predictive model of hospital admission for COPD exacerbation. Respir Care. 2015;60:1288–94.

Montserrat-Capdevila J, Godoy P, Marsal JR, Barbé F, Galván L. Risk factors for exacerbation in chronic obstructive pulmonary disease: a prospective study. Int J Tuberc Lung Dis. 2016;20:389–95.

Orea-Tejeda A, Navarrete-Peñaloza AG, Verdeja-Vendrell L, Jiménez-Cepeda A, González-Islas DG, Hernández-Zenteno R, Keirns-Davis C, Sánchez-Santillán R, Velazquez-Montero A, Puentes RG. Right heart failure as a risk factor for severe exacerbation in patients with chronic obstructive pulmonary disease: prospective cohort study. Clin Respir J. 2018;12:2635–41.

Papi A, Vestbo J, Fabbri L, Corradi M, Prunier H, Cohuet G, Guasconi A, Montagna I, Vezzoli S, Petruzzelli S, et al. Extrafine inhaled triple therapy versus dual bronchodilator therapy in chronic obstructive pulmonary disease (TRIBUTE): a double-blind, parallel group, randomised controlled trial. Lancet. 2018;391:1076–84.

Lipson DA, Barnhart F, Brealey N, Brooks J, Criner GJ, Day NC, Dransfield MT, Halpin DMG, Han MK, Jones CE, et al. Once-daily single-inhaler triple versus dual therapy in patients with COPD. N Engl J Med. 2018;378:1671–80.

Pasquale MK, Xu Y, Baker CL, Zou KH, Teeter JG, Renda AM, Davis CC, Lee TC, Bobula J. COPD exacerbations associated with the modified Medical Research Council scale and COPD assessment test among Humana Medicare members. Int J Chron Obstruct Pulmon Dis. 2016;11:111–21.

Schuler M, Wittmann M, Faller H, Schultz K. Including changes in dyspnea after inpatient rehabilitation improves prediction models of exacerbations in COPD. Respir Med. 2018;141:87–93.

Singh D, Papi A, Corradi M, Pavlišová I, Montagna I, Francisco C, Cohuet G, Vezzoli S, Scuri M, Vestbo J. Single inhaler triple therapy versus inhaled corticosteroid plus long-acting β 2 -agonist therapy for chronic obstructive pulmonary disease (TRILOGY): a double-blind, parallel group, randomised controlled trial. Lancet. 2016;388:963–73.

Søgaard M, Madsen M, Løkke A, Hilberg O, Sørensen HT, Thomsen RW. Incidence and outcomes of patients hospitalized with COPD exacerbation with and without pneumonia. Int J Chron Obstruct Pulmon Dis. 2016;11:455–65.

Stanford RH, Nag A, Mapel DW, Lee TA, Rosiello R, Schatz M, Vekeman F, Gauthier-Loiselle M, Merrigan JFP, Duh MS. Claims-based risk model for first severe COPD exacerbation. Am J Manag Care. 2018;24:e45–53.

Stanford RH, Lau MS, Li Y, Stemkowski S. External validation of a COPD risk measure in a commercial and medicare population: the COPD treatment ratio. J Manag Care Spec Pharm. 2019;25:58–69.

Vestbo J, Papi A, Corradi M, Blazhko V, Montagna I, Francisco C, Cohuet G, Vezzoli S, Scuri M, Singh D. Single inhaler extrafine triple therapy versus long-acting muscarinic antagonist therapy for chronic obstructive pulmonary disease (TRINITY): a double-blind, parallel group, randomised controlled trial. Lancet. 2017;389:1919–29.

Wei X, Ma Z, Yu N, Ren J, Jin C, Mi J, Shi M, Tian L, Gao Y, Guo Y. Risk factors predict frequent hospitalization in patients with acute exacerbation of COPD. Int J Chron Obstruct Pulmon Dis. 2018;13:121–9.

Whalley D, Svedsater H, Doward L, Crawford R, Leather D, Lay-Flurrie J, Bosanquet N. Follow-up interviews from The Salford Lung Study (COPD) and analyses per treatment and exacerbations. NPJ Prim Care Respir Med. 2019;29:20.

Zeiger RS, Tran TN, Butler RK, Schatz M, Li Q, Khatry DB, Martin U, Kawatkar AA, Chen W. Relationship of blood eosinophil count to exacerbations in chronic obstructive pulmonary disease. J Allergy Clin Immunol Pract. 2018;6:944-954.e945.

Vogelmeier CF, Kostikas K, Fang J, Tian H, Jones B, Morgan CL, Fogel R, Gutzwiller FS, Cao H. Evaluation of exacerbations and blood eosinophils in UK and US COPD populations. Respir Res. 2019;20:178.

Celli BR, Fabbri LM, Aaron SD, Agusti A, Brook R, Criner GJ, Franssen FME, Humbert M, Hurst JR, O’Donnell D, et al. An updated definition and severity classification of COPD exacerbations: the Rome proposal. Am J Respir Crit Care Med. 2021;204:1251–8.

Adir Y, Hakrush O, Shteinberg M, Schneer S, Agusti A. Circulating eosinophil levels do not predict severe exacerbations in COPD: a retrospective study. ERJ Open Research. 2018;4:00022–2018.

Bartels W, Adamson S, Leung L, Sin DD, van Eeden SF. Emergency department management of acute exacerbations of chronic obstructive pulmonary disease: factors predicting readmission. Int J Chron Obstruct Pulmon Dis. 2018;13:1647–54.

Kim V, Zhao H, Regan E, Han MK, Make BJ, Crapo JD, Jones PW, Curtis JL, Silverman EK, Criner GJ, COPDGene Investigators. The St. George’s Respiratory Questionnaire definition of chronic bronchitis may be a better predictor of COPD exacerbations compared with the classic definition. Chest. 2019;156:685–95.

Abston E, Comellas A, Reed RM, Kim V, Wise RA, Brower R, Fortis S, Beichel R, Bhatt S, Zabner J, et al. Higher BMI is associated with higher expiratory airflow normalised for lung volume (FEF25-75/FVC) in COPD. BMJ Open Respir Res. 2017;4: e000231.

Emura I, Usuda H, Satou K. Appearance of large scavenger receptor A-positive cells in peripheral blood: a potential risk factor for severe exacerbation of chronic obstructive pulmonary disease. Pathol Int. 2019;69:187–92.

Erol S, Sen E, Gizem Kilic Y, Yousif A, Akkoca Yildiz O, Acican T, Saryal S. Does the 2017 revision improve the ability of GOLD to predict risk of future moderate and severe exacerbation? Clin Respir J. 2018;12:2354–60.

Han MZ, Hsiue TR, Tsai SH, Huang TH, Liao XM, Chen CZ. Validation of the GOLD 2017 and new 16 subgroups (1A–4D) classifications in predicting exacerbation and mortality in COPD patients. Int J Chron Obstruct Pulmon Dis. 2018;13:3425–33.

Huang TH, Hsiue TR, Lin SH, Liao XM, Su PL, Chen CZ. Comparison of different staging methods for COPD in predicting outcomes. Eur Resp J. 2018;51:1700577.

Jung YH, Lee DY, Kim DW, Park SS, Heo EY, Chung HS, Kim DK. Clinical significance of laryngopharyngeal reflux in patients with chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis. 2015;10:1343–51.

CAS   PubMed   PubMed Central   Google Scholar  

Kim J, Kim WJ, Lee CH, Lee SH, Lee MG, Shin KC, Yoo KH, Lee JH, Lim SY, Na JO, et al. Which bronchodilator reversibility criteria can predict severe acute exacerbation in chronic obstructive pulmonary disease patients? Respir Res. 2017;18:107.

Kobayashi S, Hanagama M, Ishida M, Sato H, Ono M, Yamanda S, Yamada M, Aizawa H, Yanai M. Clinical characteristics and outcomes in Japanese patients with COPD according to the 2017 GOLD classification: the Ishinomaki COPD Network Registry. Int J Chron Obstruct Pulmon Dis. 2018;13:3947–55.

Lee SH, Lee JH, Yoon HI, Park HY, Kim TH, Yoo KH, Oh YM, Jung KS, Lee SD, Lee SW. Change in inhaled corticosteroid treatment and COPD exacerbations: an analysis of real-world data from the KOLD/KOCOSS cohorts. Respir Res. 2019;20:62.

Pavlovic R, Stefanovic S, Lazic Z, Jankovic S. Factors associated with the rate of COPD exacerbations that require hospitalization. Turk J Med Sci. 2017;47:134–41.

Song JH, Lee CH, Um SJ, Park YB, Yoo KH, Jung KS, Lee SD, Oh YM, Lee JH, Kim EK, Kim DK. Clinical impacts of the classification by 2017 GOLD guideline comparing previous ones on outcomes of COPD in real-world cohorts. Int J Chron Obstruct Pulmon Dis. 2018;13:3473–84.

Sundh J, Johansson G, Larsson K, Lindén A, Löfdahl CG, Sandström T, Janson C. The phenotype of concurrent chronic bronchitis and frequent exacerbations in patients with severe COPD attending Swedish secondary care units. Int J Chron Obstruct Pulmon Dis. 2015;10:2327–34.

Urwyler P, Hussein NA, Bridevaux PO, Chhajed PN, Geiser T, Grendelmeier P, Zellweger LJ, Kohler M, Maier S, Miedinger D, et al. Predictive factors for exacerbation and reexacerbation in chronic obstructive pulmonary disease: an extension of the Cox model to analyze data from the Swiss COPD cohort. Multidiscip Respir Med. 2019;14:7.

Wallace AE, Kaila S, Bayer V, Shaikh A, Shinde MU, Willey VJ, Napier MB, Singer JR. Health care resource utilization and exacerbation rates in patients with COPD stratified by disease severity in a commercially insured population. J Manag Care Spec Pharm. 2019;25:205–17.

Halpin DMG, Decramer M, Celli BR, Mueller A, Metzdorf N, Tashkin DP. Effect of a single exacerbation on decline in lung function in COPD. Respir Med. 2017;128:85–91.

Bade BC, DeRycke EC, Ramsey C, Skanderson M, Crothers K, Haskell S, Bean-Mayberry B, Brandt C, Bastian LA, Akgün KM. Sex differences in veterans admitted to the hospital for chronic obstructive pulmonary disease exacerbation. Ann Am Thorac Soc. 2019;16:707–14.

Iyer AS, Bhatt SP, Dransfield M, Kinney G, Holm K, Wamboldt FS, Hanania N, Martinez C, Regan E, Foreman MG, et al. Psychological distress prospectively predicts severe exacerbations in smokers with and without airflow limitation—a longitudinal follow-up study of the COPDGene cohort [abstract]. Am J Respir Crit Care Med. 2017. https://doi.org/10.1164/ajrccm-conference.2017.195.1_MeetingAbstracts.A4709 .

Diamond M, Zhao H, Armstrong HF, Morrison M, Bailey KL, Carretta EE, Criner GJ, Han MK, Bleeker E, Cooper CB, et al. Anxiety and depression, either alone or in combination, are associated with respiratory exacerbations in smokers with and without COPD [abstract]. Am J Respir Crit Care Med. 2017;195:1615–31.

Google Scholar  

Lau CS, Siracuse BL, Chamberlain RS. Readmission After COPD Exacerbation Scale: determining 30-day readmission risk for COPD patients. Int J Chron Obstruct Pulmon Dis. 2017;12:1891–902.

Pikoula M, Quint JK, Nissen F, Hemingway H, Smeeth L, Denaxas S. Identifying clinically important COPD sub-types using data-driven approaches in primary care population based electronic health records. BMC Med Inform Decis Mak. 2019;19:86.

Wei YF, Tsai YH, Wang CC, Kuo PH. Impact of overweight and obesity on acute exacerbations of COPD—subgroup analysis of the Taiwan Obstructive Lung Disease cohort. Int J Chron Obstruct Pulmon Dis. 2017;12:2723–9.

Barnes PJ, Celli BR. Systemic manifestations and comorbidities of COPD. Eur Resp J. 2009;33:1165–85.

Polverino E, Dimakou K, Hurst J, Martinez-Garcia MA, Miravitlles M, Paggiaro P, Shteinberg M, Aliberti S, Chalmers JD. The overlap between bronchiectasis and chronic airway diseases: state of the art and future directions. Eur Respir J. 2018;52:1800328.

Xu W, Collet JP, Shapiro S, Lin Y, Yang T, Platt RW, Wang C, Bourbeau J. Independent effect of depression and anxiety on chronic obstructive pulmonary disease exacerbations and hospitalizations. Am J Respir Crit Care Med. 2008;178:913–20.

Chapman KR, Hurst JR, Frent SM, Larbig M, Fogel R, Guerin T, Banerji D, Patalano F, Goyal P, Pfister P, et al. Long-term triple therapy de-escalation to indacaterol/glycopyrronium in patients with chronic obstructive pulmonary disease (SUNSET): a randomized, double-blind, triple-dummy clinical trial. Am J Respir Crit Care Med. 2018;198:329–39.

Couillard S, Larivée P, Courteau J, Vanasse A. Eosinophils in COPD exacerbations are associated with increased readmissions. Chest. 2017;151:366–73.

Ferguson GT, Rabe KF, Martinez FJ, Fabbri LM, Wang C, Ichinose M, Bourne E, Ballal S, Darken P, DeAngelis K, et al. Triple therapy with budesonide/glycopyrrolate/formoterol fumarate with co-suspension delivery technology versus dual therapies in chronic obstructive pulmonary disease (KRONOS): a double-blind, parallel-group, multicentre, phase 3 randomised controlled trial. Lancet Respir Med. 2018;6:747–58.

Ko FWS, Chan KP, Ngai J, Ng SS, Yip WH, Ip A, Chan TO, Hui DSC. Blood eosinophil count as a predictor of hospital length of stay in COPD exacerbations. Respirology. 2019;25:259–66.

MacDonald MI, Osadnik CR, Bulfin L, Hamza K, Leong P, Wong A, King PT, Bardin PG. Low and high blood eosinophil counts as biomarkers in hospitalized acute exacerbations of COPD. Chest. 2019;156:92–100.

Müllerová H, Hahn B, Simard EP, Mu G, Hatipoğlu U. Exacerbations and health care resource use among patients with COPD in relation to blood eosinophil counts. Int J Chron Obstruct Pulmon Dis. 2019;14:683–92.

Bafadhel M, Greening NJ, Harvey-Dunstan TC, Williams JEA, Morgan MD, Brightling CE, Hussain SF, Pavord ID, Singh SJ, Steiner MC. Blood eosinophils and outcomes in severe hospitalised exacerbations of COPD. Chest. 2016;150:320–8.

Roche N, Chapman KR, Vogelmeier CF, Herth FJF, Thach C, Fogel R, Olsson P, Patalano F, Banerji D, Wedzicha JA. Blood eosinophils and response to maintenance chronic obstructive pulmonary disease treatment. Data from the FLAME trial. Am J Respir Crit Care Med. 2017;195:1189–97.

Vestbo J, Vogelmeier CF, Small M, Siddall J, Fogel R, Kostikas K. Inhaled corticosteroid use by exacerbations and eosinophils: a real-world COPD population. Int J Chron Obstruct Pulmon Dis. 2019;14:853–61.

Watz H, Tetzlaff K, Wouters EFM, Kirsten A, Magnussen H, Rodriguez-Roisin R, Vogelmeier C, Fabbri LM, Chanez P, Dahl R, et al. Blood eosinophil count and exacerbations in severe chronic obstructive pulmonary disease after withdrawal of inhaled corticosteroids: a post-hoc analysis of the WISDOM trial. Lancet Respir Med. 2016;4:390–8.

Pavord ID, Lettis S, Locantore N, Pascoe S, Jones PW, Wedzicha JA, Barnes NC. Blood eosinophils and inhaled corticosteroid/long-acting beta-2 agonist efficacy in COPD. Thorax. 2016;71:118–25.

Singh D. Predicting corticosteroid response in chronic obstructive pulmonary disease. Blood eosinophils gain momentum. Am J Respir Crit Care Med. 2017;196:1098–100.

Vestbo J, Prescott E, Almdal T, Dahl M, Nordestgaard BG, Andersen T, Sørensen TIA, Lange P. Body mass, fat-free body mass, and prognosis in patients with chronic obstructive pulmonary disease from a random population sample: findings from the Copenhagen City Heart Study. Am J Respir Crit Care Med. 2006;173:79–83.

Agustí AGN, Noguera A, Sauleda J, Sala E, Pons J, Busquets X. Systemic effects of chronic obstructive pulmonary disease. Eur Respir J. 2003;21:347–60.

Agustí AGN, Sauleda J, Miralles C, Gomez C, Togores B, Sala E, Batle S, Busquets X. Skeletal muscle apoptosis and weight loss in chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2002;166:485–9.

Labaki WW, Martinez FJ. Time to understand the infrequency of the frequent exacerbator phenotype in COPD. Chest. 2018;153:1087–8.

Hartley BF, Barnes NC, Lettis S, Compton CH, Papi A, Jones P. Risk factors for exacerbations and pneumonia in patients with chronic obstructive pulmonary disease: a pooled analysis. Respir Res. 2020;21:5.

Kim Y, Kim YJ, Kang YM, Cho WK. Exploring the impact of number and type of comorbidities on the risk of severe COPD exacerbations in Korean Population: a Nationwide Cohort Study. BMC Pulm Med. 2021;21:151.

Mackay AJ, Kostikas K, Roche N, Frent SM, Olsson P, Pfister P, Gupta P, Patalano F, Banerji D, Wedzicha JA. Impact of baseline symptoms and health status on COPD exacerbations in the FLAME study. Respir Res. 2020;21:93.

Smulders L, van der Aalst A, Neuhaus EDET, Polman S, Franssen FME, van Vliet M, de Kruif MD. Decreased risk of COPD exacerbations in obese patients. COPD. 2020;17:485–91.

Battisti WP, Wager E, Baltzer L, Bridges D, Cairns A, Carswell CI, Citrome L, Gurr JA, Mooney LA, Moore BJ, et al. Good publication practice for communicating company-sponsored medical research: GPP3. Ann Intern Med. 2015;163:461–4.

Putcha N, Barr RG, Han M, Woodruff PG, Bleecker ER, Kanner RE, Martinez FJ, Tashkin DP, Rennard SI, Breysse P, et al. Understanding the impact of passive smoke exposure on outcomes in COPD [abstract]. Am J Respir Crit Care Med. 2015;191:411–20.

Wu Z, Yang D, Ge Z, Yan M, Wu N, Liu Y. Body mass index of patients with chronic obstructive pulmonary disease is associated with pulmonary function and exacerbations: a retrospective real world research. J Thorac Dis. 2018;10:5086–99.

Download references

Acknowledgements

Medical writing support, under the direction of the authors, was provided by Julia King, PhD, and Sarah Piggott, MChem, CMC Connect, McCann Health Medical Communications, funded by AstraZeneca in accordance with Good Publication Practice (GPP3) guidelines [ 109 ].

This study was supported by AstraZeneca.

Author information

Authors and affiliations.

UCL Respiratory, University College London, London, WC1E 6BT, UK

John R. Hurst

Division of Pulmonary and Critical Care, University of Michigan, Ann Arbor, MI, USA

MeiLan K. Han

Formerly of Parexel International, Mohali, India

Barinder Singh, Gagandeep Kaur & Mohd Kashif Siddiqui

Parexel International, Mohali, India

Sakshi Sharma

Formerly of AstraZeneca, Cambridge, UK

Enrico de Nigris

AstraZeneca, Gothenburg, Sweden

Ulf Holmgren

You can also search for this author in PubMed   Google Scholar

Contributions

The authors have made the following declaration about their contributions. JRH and MKH made substantial contributions to the interpretation of data; BS, SS, GK, and MKS made substantial contributions to the acquisition, analysis, and interpretation of data; EdN and UH made substantial contributions to the conception and design of the work and the interpretation of data. All authors contributed to drafting or critically revising the article, have approved the submitted version, and agree to be personally accountable for their own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. All authors read and approved the final manuscript.

Corresponding author

Correspondence to John R. Hurst .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

JRH reports consulting fees from AstraZeneca; speaker fees from AstraZeneca, Chiesi, Pfizer, and Takeda; and travel support from GlaxoSmithKline and AstraZeneca. MKH reports assistance with conduction of this research and publication from AstraZeneca; personal fees from Aerogen, Altesa Biopharma, AstraZeneca, Boehringer Ingelheim, Chiesi, Cipla, DevPro, GlaxoSmithKline, Integrity, Medscape, Merck, Mylan, NACE, Novartis, Polarean, Pulmonx, Regeneron, Sanofi, Teva, Verona, United Therapeutics, and UpToDate; either in kind research support or funds paid to the institution from the American Lung Association, AstraZeneca, Biodesix, Boehringer Ingelheim, the COPD Foundation, Gala Therapeutics, the NIH, Novartis, Nuvaira, Sanofi, and Sunovion; participation in Data Safety Monitoring Boards for Novartis and Medtronic with funds paid to the institution; and stock options from Altesa Biopharma and Meissa Vaccines. BS, GK, and MKS are former employees of Parexel International. SS is an employee of Parexel International, which was funded by AstraZeneca to conduct this analysis. EdN is a former employee of AstraZeneca and previously held stock and/or stock options in the company. UH is an employee of AstraZeneca and holds stock and/or stock options in the company.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file1: table s1..

Search strategies. Table S2. List of included studies with linked publications. Table S3. Study characteristics across the 76 included studies. Table S4. Clinical characteristics of the patients assessed across the included studies.

Additional file 2: Fig. S1.

Sex (male vs female) as a risk factor for moderate-to-severe exacerbations. Fig. S2. Sex (male vs female) as a risk factor for severe exacerbations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hurst, J.R., Han, M.K., Singh, B. et al. Prognostic risk factors for moderate-to-severe exacerbations in patients with chronic obstructive pulmonary disease: a systematic literature review. Respir Res 23 , 213 (2022). https://doi.org/10.1186/s12931-022-02123-5

Download citation

Received : 02 March 2022

Accepted : 20 July 2022

Published : 23 August 2022

DOI : https://doi.org/10.1186/s12931-022-02123-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Exacerbations
• Comorbidities
• Hospitalization

Respiratory Research

ISSN: 1465-993X

• General enquiries: [email protected]

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

The Power of Peer Groups (and How to Start One)

• Dan Richards

Accelerate your personal and professional development.

A peer group is a group of professionals who meet on a regular basis to confidentially share their current issues and exchange insights that will help each member excel. Their benefits have been proven by successful leaders and workers for centuries. If you want to form a peer group of your own, take the following steps:

• Recruit the right members: Try to form a group of six to 10 people with diverse personal and professional backgrounds. Make sure you each respect one another, and are committed to meeting regularly throughout the year.
• Set guidelines. Agree upon how often you’ll meet, who will organize the agenda for your meetings, as well as who will moderate and keep time.
• Agree on values. Confidentiality, candor, empathy, and balance are required for your group to establish an inclusive and productive culture.
• Check in to keep up the momentum. Every few months, ask members for feedback to learn if the group dynamic is still beneficial for each member.

Have you ever wished you had a community to support you throughout the long journey of your career? I’m not referring to a mentor who provides you with occasional guidance or a network to lean on when times get rough. I’m talking about a group of people to regularly bounce ideas off of, advise you on tough decisions, and offer guidance when stressful work challenges arise.

• DR Dan Richards is a serial founder, and as a former CEO, he led the turnaround of a public company in the financial industry. Today, he is an award-winning faculty member at the Rotman School of Management, where the MBA class of 2022 voted him Best Professor. He oversees the course associated with MBA student internships. You can find him on LinkedIn.

Partner Center

• Open access
• Published: 09 November 2023

People-centred primary health care: a scoping review

• Resham B. Khatri 1 , 2 ,
• Eskinder Wolka 3 ,
• Frehiwot Nigatu 3 ,
• Anteneh Zewdie 3 ,
• Daniel Erku 4 , 5 ,
• Aklilu Endalamaw 1 , 6 &
• Yibeltal Assefa 1  

BMC Primary Care volume  24 , Article number:  236 ( 2023 ) Cite this article

2199 Accesses

11 Altmetric

Metrics details

Integrated people-centred health services (IPCHS) are vital for ensuring comprehensive care towards achieving universal health coverage (UHC). The World Health Organisation (WHO) envisions IPCHS in delivery and access to health services. This scoping review aimed to synthesize available evidence on people-centred primary health care (PHC) and primary care.

We conducted a scoping review of published literature on people-centred PHC. We searched eight databases (PubMed, Scopus, Embase, CINAHL, Cochrane, PsycINFO, Web of Science, and Google Scholar) using search terms related to people-centred and integrated PHC/primary care services. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) checklist to select studies. We analyzed data and generated themes using Gale's framework thematic analysis method. Themes were explained under five components of the WHO IPCHS framework.

A total of fifty-two studies were included in the review; most were from high-income countries (HICs), primarily focusing on patient-centred primary care. Themes under each component of the framework included: engaging and empowering people and communities (engagement of community, empowerment and empathy); strengthening governance and accountability (organizational leadership, and mutual accountability); reorienting the model of care (residential care, care for multimorbidity, participatory care); coordinating services within and across sectors (partnership with stakeholders and sectors, and coordination of care); creating an enabling environment and funding support (flexible management for change; and enabling environment).

Conclusions

Several people-centred PHC and primary care approaches are implemented in HICs but have little priority in low-income countries. Potential strategies for people-centred PHC could be engaging end users in delivering integrated care, ensuring accountability, and implementing a residential model of care in coordination with communities. Flexible management options could create an enabling environment for strengthening health systems to deliver people-centred PHC services.

Peer Review reports

Introduction

The concept of “integrated and people-centred care” comprises two overarching concepts: integrated and people-centred care. The first concept, integrated care, is advanced from conventional illnesses-oriented and disease-focused health care. Illnesses-oriented care focuses on illness and cure, episodic consultation, and users as consumers purchase care. In contrast, disease-focused care refers to the management of diseases and priority disease control interventions, including their risk factors [ 1 ]. Additionally, integrated care means putting people and communities (not diseases), at the centre of health systems and empowering people and communities to take charge of their health by ensuring well-coordinated care around their needs, responding to fragmentations of care, and improving quality and cost-effectiveness rather than being passive recipients of services [ 1 , 2 ].

Furthermore, integrated care emphasizes holistic care to improve population health and wellbeing with continued care across the life course, around needs with shared responsibility and accountability [ 3 ]. Ensuring integrated care empowers people to tackle the determinants of ill-health through systems thinking and partnerships, encouraging them to become co-producers of care in multilevel (individual, organizational and policy) systems [ 3 ]. Thus, integrated care is best understood as a set of practices intricately shaped by contextual factors to improve health status, and reduce morbidities and mortalities [ 4 ].

Moreover, the second concept, i.e., people-centred care (PCC) is derived from patient and person-centred care. In the late 1960s, patient-centred care (different from illness-oriented care) was introduced and continued for several decades, opposing previously prevailing bio-medically oriented and paternalistic views of healthcare [ 5 ]. Patient-centred care aims to make a functional life, affirming the ethical principles of respect for persons and justice, striving to make the health system more responsive to the health services needs [ 5 , 6 ]. Advocates of market solutions to healthcare have been adopting patient-centred care by arguing for improved flexibility of consumer-oriented health care options and enhancing individual choice [ 7 ]. In contrast, person-centred care refers to caring for a meaningful life, and is a further development of patient-centred care based on personal philosophy, where the person denotes human and distinguishes from everything else [ 5 ]. Primarily, PCC is an expansion of patient-centred/person-centred care where people are involved in a care cycle, including the public, healthcare practitioners, and care organizations or systems. The PCC focuses on organizing principles for integrated care as a service innovation relating to individual service users, families and concerned communities [ 2 ]. Transforming the health care system towards people-centred health care requires action at four levels of the system: i) individuals, families and communities; ii) care providers; iii) health organizations; and iv) health systems [ 8 ]. The PCC is associated with better care continuity, considered care delivery by frontline workers within the health system, and responsive care practices and service utilization [ 9 , 10 ].

The World Health Organization’s (WHO) Framework on integrated people-centred health services (IPCHS) combines the concepts of integrated care and people-centred care [ 11 ]. The framework envisions that all people have equal access to quality health services, co-produces health care to meet their health needs across the life course and respect their preferences, and coordinated and quality care (comprehensive, safe, effective, timely, efficient, and acceptable) along the continuum by all skilled and motivated carers and work in a supportive environment [ 11 ]. The conceptualization of integrated PCC puts people’s needs first in designing and delivering health services with principles of quality, safety, longitudinality (duration and depth of contact), closeness to communities, and responsive care (equity in access, quality, responsiveness and participation, efficiency, and resilience) [ 12 ]. Specifically, the WHO framework on IPCHS outlines five interwoven strategies for management and health service delivery: engaging and empowering people and communities; strengthening governance and accountability; reorienting the model of care; coordinating services within and across sectors; and creating an enabling environment and funding support [ 13 , 14 ].

Primary health care (PHC) is a whole-of-society approach to organize and strengthen national health systems to bring health services closer to communities. The PHC approach comprises integrated health services to meet people’s health needs throughout their lives, addressing the broader determinants of health through multisectoral actions and empowering communities to improve health [ 15 ]. While primary care is a first level of care, it is usually delivered from prehospital, peripheral health facilities, and community settings [ 3 ]. People-centred PHC is the foundation of health systems that prioritize people first and have the potential to address diverse health needs by putting people and communities at the center of the system, empowering personalized health decision-making, and adapting health services to the local socio-cultural context [ 16 ]. Current body of literature focuses on people-centred integrated health services, especially medical care in hospitals, or family medicine or care by general practitioners. Nonetheless, there is a dearth of research that synthesize standalone studies on people-centred PHC and primary care using the WHO’s IPCHS framework. Thus, this study aimed to synthesize evidence on people-centred PHC interventions and strategies, their issues, and challenges. The findings of this review could inform strategies for strengthening the health system towards people-centredness in PHC systems and delivery and utilization of services.

This study is a scoping review of the literature reporting people-centred PHC services/ primary care. A scoping review method helps to synthesize and analyze existing literature on a topic and map the scope of available evidence. The process involves six steps: identifying the research question; identifying relevant studies, selecting studies; charting data; collating, summarizing, and reporting results; and consultation (optional) [ 17 , 18 ]. We employed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) checklist to support comprehensive reporting of methods and findings (Supplementary Information, Table S 1 ) [ 19 , 20 ].

Identifying the research question

We identified the research question focusing on people-centred PHC/primary care services. The key research question was to review and synthesize the evidence on issues and challenges related to people-centredness in PHC/primary care services. We brainstormed on two concepts: people-centred care and PHC/primary care. These concepts guided identifying search terms under each concept and developing search strings. Our research team assumed that the proposed research question is broad to provide a breadth of issues to be explored in the review. The research question was further clarified by preliminary discussion among authors and agreed on the scope and significance of the topic.

Identifying relevant studies

We searched eight databases (PubMed, Scopus, Embase, CINAHL, Cochrane, PsycINFO, Web of Science, and Google Scholar). The search strategy was built on two key concepts and related search terms: People-centred care (patient centred care, people centred care, person centred care, patient-centred care, people-centred care, person-centred care, patient centered care, people centered care, person centered care, patient-centered care, people-centered care, client-centered care, client centered care, person-centered care); Primary Health Care (primary health care, public health care, community care, primary care, primary care nursing, family medicine, family practice, general practice) on each database. Boolean operators (AND/OR) and truncations (“”, *) varied depending on the database. The search included all studies published in English until 30 January 2023 (no starting date was applied in the search). No country-related limitations were applied.

Selection of studies

We included all studies that dealt with PCC regardless of their designs. Based on the title and abstract, screening was undertaken initially by the first author and further assessed by the second author. This was followed by a full-text screening initially by the first author and evaluated by the second and third authors. Any disagreements were resolved by discussion with the last author. We applied some post hoc inclusion and exclusion criteria based on the research question and new topic familiarity through reading the studies. For example, we included studies considering the population (health service users, care providers and managers), concept (PCC/integrated care), and contexts (PHC and primary care systems) of the study [ 21 ]. We included studies if their findings can answer our review question rather than the quality of individual studies. We followed the standard scoping review PRISMA-ScR checklist [ 19 , 22 ] and took reference to previous scoping reviews [ 23 , 24 ]. The included studies are based on the findings and their interpretation rather than the inclusion criteria [ 25 , 26 ].

Charting the data

A data-charting form was developed to extract data from each study covering author, year, country, type of study, key concepts, and main findings (Supplementary file, Table S 2 ). Data were extracted by the first and double-checked by the second and last authors.

Collating, summarizing, and reporting results

The first author did data analysis with guidance and support from the last author. Thematic analysis of data was conducted by adopting Gale’s framework method [ 27 ]. This analysis method adopts multiple steps such as collection of raw data (main findings about the research question for this review), familiarisation with data, paraphrasing of data/label according to the nature of data, developing/applying the analytical framework, charting data into the framework matrix, and finally interpretation. After reading and familiarisation the data, we extracted important concepts/categories and grouped them (with similar ideas) into the five components (engaging and empowering people and communities; strengthening governance and accountability; reorienting the model of care; coordinating services within and across sectors; creating an enabling environment and funding support) of the WHO ICPHS framework. Within each component, themes were generated by grouping similar categories/ideas and concepts. Findings were reported in three forms; first, outcomes of database search results were presented in the flow chart. Second, a customized summary of the data charting table (covering the author, location, and key ideas related to the research question) was presented. Finally, generated themes were explained and interpreted in the narrative paragraphs under each component of the analytical framework.

A search yielded 4494 records from all databases (Fig.  1 ). We removed 2090 duplicated records. Then, studies were screened for relevance based on title and abstract, whereby 2321 were excluded, leaving 83 studies for full-text screening. A further 31 studies were excluded after the full-text screening with reasons. A further 52 studies were included in the final review.

PRISMA-ScR flow chart showing the selection of studies for the review

Overview of included studies and generated themes

Table 1 presents an overview of studies included in the review, including countries where studies were undertaken, and each study mapped with generated themes. Of 52 studies, 39 were from high-income countries (HICs): 19 studies were from the USA, eight studies were from Canada (4) and the Netherlands (4), six studies were from the UK (3) and Australia (3), four studies were from Norway (2) and Sweden (2), and one each from Greece and Finland. Two studies were from upper-middle-income countries (UMICs), including one from multi-country (Latin America), Mexico, and South Africa. Three studies were from low-and lower-middle-income countries (LMICs), including one from Nigeria and two from Uganda, and seven studies were not specified. Of five strategic components of the ICPHS framework (Table 1 ), 26 studies described engaging and empowering people and communities (light black); 32 studies included strengthening governance and accountability (blue); 45 studies explained reorienting the model of care (pink); 34 studies incorporated coordinating services within and across sectors (yellow); and 33 studies discussed creating an enabling environment and support for funding (green).

Engaging and empowering people and communities

Empowering and engaging people provides the opportunity, skills and resources needed to be articulated and empowered end users of health services and advocates for a reformed health system [ 13 ]. Two themes are described in this component: community engagement in health care (11 studies) and empathic empowerment (17 studies) related to PHC/primary care services.

Community engagement

The use of the PCC practices facilitated an ongoing relationship between end users (providers and service users) through a team relationship, timely communication, and care plan (e.g., enhanced coordinated, comprehensive care) [ 29 , 33 ]. Additionally, peers promoted norms to other service users and worked together to improve routine care practices [ 31 , 34 ]. Reinforcement of people engagement and positive perception improved people-centred primary and hospital care [ 41 , 54 , 59 ]. Contextual factors influencing PCC included perceptions of involvement, engagement, and co-creation of processes to achieve physical and social well-being for persons with multimorbidity [ 50 , 51 , 71 ]. Nonetheless, only the positive perception of the public filled the limited expectations, and service users’ voices were less incorporated, which decreased the traditional system's authority [ 41 , 53 ].

Empowerment and empathy

Empathy, communication in multicultural languages, people’s involvement in making decisions and designing and implementing solutions effectively empower individualized care for their health [ 57 , 59 , 61 ]. Empathic support, communication with their doctors, understanding of problems, providers’ skills, and management plans were required for improved satisfaction and the effect of interpersonal care [ 61 , 62 , 70 ]. Communication between service users and providers enhanced high perceived empathy in consultation, trust in relationships, and positive experiences and satisfaction [ 50 , 62 ]. The trusted relationships with providers, their involvement in care treatment decision-making, and emotional support from family and friends found effective people-centredness in care delivery [ 45 , 67 , 68 , 77 ]. Local arrangements for service integration, multi-professional teams, and co-location also supported building relationships for community empowerment [ 75 , 79 ].

Nonetheless, weak communication between individuals and practitioners was reflected in daily care practice activities and had poor priority in service delivery [ 73 , 75 ]. Factors of poor empowerment in care continuity were lack of flexible design, high administrative tasks, limited appointment time, poor autonomy, and unavailability of providers [ 56 , 64 ]. Several factors, such as intra- and interpersonal (e.g., perceived reluctance to engage in care), and organizational (e.g., limited encounter time, lack of discussion, psychological issues with health workers), also influenced understating the problems and health service needs [ 60 , 62 , 78 ].

Strengthening governance and accountability

Strengthening governance requires a participatory approach to policy formulation, decision-making and performance evaluation at multilevel health systems, from policy-making to the service delivery level [ 13 ].Two themes under this component were: organizational leadership (18 studies) and mutual accountability (15 studies).

Organizational leadership

Strategies to strengthen organisational leadership, human capital, and facilitating adaptive culture and innovation contributed to innovative PCC primary care services [ 32 ]. For instance, the people-centred medical home (PCMH) model created an enabling environment for delivering quality care, reduced care costs, and organizational needs, and incorporated people’s voices into governance and accountability for operations [ 36 , 53 , 59 , 63 ]. The role of professional councils (e.g., nursing, or general practitioners’ organizations) could be instituted to measure people-centredness for the implementation of PHC [ 58 , 76 ]. Similarly, increased local leadership, team communication, and high physician engagement with service users facilitated the successful implementation of people-centredness in PHC [ 37 , 46 ]. Multiple stakeholders offered an opportunity for reform and gaining an inclusive vision of PCC in Uganda [ 57 ] and Greece [ 52 ]. Furthermore, the use of digital technology supported the functionality of clinical information aligning with organizational support, availability of community resources, clinician interactions, and gap payment funding models to incentivize care workers [ 42 , 63 , 67 ].

However, the people-centred integrated care process failed to identify long-term goals, provide shared long-term care, and monitor and evaluate health care delivery for people with multimorbidity [ 47 ]. Furthermore, organizational and policy impediments (e.g., state decisional capacity laws and financial crisis), lack of documentation or low priority also impacted the delivery of integrated PCC [ 35 , 39 , 52 , 78 ].

Mutual accountability

Understanding the health system and integrating different dimensions of care ensured the changing needs of people with complex chronic illnesses [ 49 , 54 ]. Integrated responsiveness and relative priority for the cultural change improved client and professional interaction towards organized care [ 54 , 59 , 63 , 69 ]. The government policy in health system organizations assessed incentives for care coordination to meet complex needs [ 36 , 41 , 78 ]. Innovations and people-centeredness shaped the access to health facilities, costs, users’ perceived quality of care and expectations, and availability of free services [ 53 , 62 ]. Furthermore, approaching interpersonal and coordinated multidisciplinary teamwork, consultation on preventive and promotive measures supported people receiving treatment, medical information, and skill mix care practice towards people-centred holistic care [ 60 , 66 , 71 , 74 ].

Nonetheless, fragmentation, segmentation, limited funding, insufficient coverage, poor quality, ageing and chronic conditions, and lack of effectiveness and sustainability were multilevel challenges to achieving mutual accountability towards PCC [ 68 , 72 ]. Other influencing factors of mutual accountability were limited understanding of professional identities, roles, and responsibilities in continuity of care and service integration [ 75 ].

Reorienting the model of care

Reorienting the model of care means ensuring that efficient and effective health care services are designed, purchased and provided through innovative models of care that prioritize primary and community care services and the co-production of health [ 13 ]. Three themes generated in this component were: residential and home-based care (11 studies), care for people living with multiple chronic conditions or multimorbid conditions (21 studies), and participatory care (30 studies).

Residential and home-based care

The residential model of care, known as the patient-centred Medical Home (PCMH), is a new form of transformation in healthcare that offers an interprofessional model by connecting services and management in a primary care setting [ 28 , 36 , 61 , 79 ]. The centrepiece of transformation for primary care in a residential health care model restored confidence in quality of care and resulted in reduced care costs of hospital-related outcomes [ 36 , 63 ]. Such a model that was developed in iterative phases (e.g., planning, acting, observing, supporting and transforming care practices) met the needs of people’s priorities, improved holistic and more people-centred care in primary care, and addressed the health needs of disadvantaged communities [ 29 , 38 , 53 , 74 ]. The residency-affiliated community group family medicine provided goal-directed care for people with complex health problems (functioning, social situation, support and empowerment, and care satisfaction). It ensured self-management at home (e.g., engaging with nurses, telehealth, medication plans, and interactions) [ 29 , 35 , 74 , 78 ]. Positive effects of functional residential care improved satisfaction, informal and formal caregivers in clinical screening as high-risk groups, and delayed placement [ 63 , 79 ]. However, home-based residential care was unsuitable for managing several disease-based care pathways and specialist care to address individuals' needs for people with substantial comorbidity [ 35 ].

Care for people with comorbidities

PCC brought the management of chronic diseases to a new dimension of care (legitimizing the illness experience, acknowledging service users’ expertise, offering hope and providing advocacy) [ 30 ]. People with multimorbidity viewed PCC as a well-coordinated, respectful, supportive care long-term management of medical problems and prevention and promotion through behaviour change interventions [ 36 , 60 , 62 ]. Approaches of PCC in the management of comorbidities were effective communication, information, knowledge sharing, understanding demonstration of provider’s multidimensional skills, and agreement about treatment plans [ 39 , 57 , 69 ]. Additionally, care from non-physicians found important in comorbidities; for instance, pharmacists provided direct care services, ensured access to community resources, assisted care transitions, and provided interprofessional education [ 28 ]. Diabetes specialist nurses expressed needs that diverge and converge for people with diabetes [ 56 ]. Developing training for health care providers for self-management interventions and self-care practices positively impacted people with chronic diseases [ 42 , 48 ]. Such care practices brought the benefit of clinical care, active involvement in care, and shifting from disease-focused to people-centred PHC [ 34 , 35 , 49 , 74 ]. Furthermore, the Family Health Team and multidisciplinary providers pursued continuity and care coordination, allowing site-specific program implementation and commitment to timely delivery of health services [ 37 , 40 ]. Interdisciplinary teams and informal caregivers enabled people-centred medication therapy for older people, management services with continuous quality improvement initiatives, and inpatient family medicine service [ 28 , 63 , 78 ]. Nevertheless, understanding variations between GP practices and poor documentation of records of people-centeredness had challenges in applying evidence-based medicine [ 39 , 43 ]. Additionally, some of the populations (e.g., migrants and refugees) were neglected in the management of chronic disease, had gaps in irregular care and providers, lack of information (medical history to solve health problems), and limited time spent with the people [ 41 , 52 , 78 ].

Participatory model of care

Designing participatory and holistic or whole-person care (e.g., respect and value, choice, dignity, self-determination, purposeful living), and had the potential to address multiple dimensions of care for wellbeing (e.g., physical, mental, and social needs) by knowing and confirming tailored health plan, inter-professional teamwork, and care provision in collaboration with families [ 31 , 44 , 51 , 52 , 64 , 66 , 74 , 77 ]. Understanding the participatory approach of PCC informed quality of care (e.g., availability of medication, shorter waiting times, flexible facility opening hours, courteous health workers) for care for ageing problems [ 62 , 65 , 70 ]. The care process for people with social and health complexity (for health needs of older adults, and referral practice) was found effective in primary care to adapt peoples’ preferences [ 54 , 70 ]. There was effective communication by using electronic health records to people with complex health issues that supported the involvement of people and families in health care (e.g., practice set-up, planning, and change in consultation) [ 39 , 45 , 49 , 57 , 69 , 72 ]. Participation of people built trust through shifting the role of self-care based on medical knowledge and pragmatic experience of engagement in care process, and ensuring provider relationship and guidance [ 33 , 34 ]. Participation of service users (e.g., obtaining feedback, engaging stakeholders, adapting PCC quality improvement for better quality care) improved service integration and practices [ 42 , 75 , 77 ].

Participatory and coordinated care enhanced joint working, fostering communication and professional cultures (shared beliefs and values) by exploring and prioritizing the problems (e.g., knowing the person, identifying problems, prioritizing care, treatment, evaluating decisions and implementation) [ 72 , 73 , 75 ]. Approaches to co-design and co-creation built trust, partnering with professionals and users, communities, and individuals experience [ 51 , 66 , 74 , 77 ]. Strategies of participatory care included evidence-based decision practice, enhancing interdisciplinary team approach to continuity of care, developing training for providers, involvement of people in sharing experience (e.g., empathy in consultation, physical and social wellbeing), and providers' attitudes (open communication, caring behaviours) [ 48 , 50 , 51 , 54 , 58 ]. Furthermore, system responsiveness for quality care (e.g., affordable, coordinated, accessible) moved towards the long-term goal of universal access [ 38 , 47 , 72 ]. However, challenges such as the unavailability of family physicians, limited information and communication technology, and heterogeneity of people-centred quality improvement influenced the integrated people-centred primary care among disadvantaged populations (e.g., refugees) [ 52 , 55 ]. In some public facilities, the care process was unseen and disrespected, lacking continuity, transition, and coordinated care [ 61 , 68 ].

Coordinating services within and across sectors

Coordination requires integrating care providers within and across health care settings, developing referral systems and networks among levels of care, and creating linkages between health and other sectors [ 13 ]. Two themes were described under this component: partnership with stakeholders and sectors (24 studies) and coordination for quality care (14 studies).

Partnership with stakeholders and sectors

Partnership with other sectors supports engagement in people-centred PHC. Involvement of stakeholders and sectors (e.g., trust, understanding of purpose, clarity of expectations, and power-sharing) facilitated priorities for care evaluation and treatment outcomes [ 71 , 74 , 76 ]. Developing partnerships and team-based approaches (appointment tool guide communication) with people experiencing complex diseases to reduce stigma, social and relational integration for care coordination, and self-management [ 35 , 36 , 64 , 77 , 79 ].

Communication technology support partnership with other sectors. Communication technology and resources support non-physician healthcare providers [ 38 , 78 ]. Integrated health information technology was perceived as effective in the organization and management of chronic diseases, including the medical and care needs (discharge-related information sent from the hospital and care providers linking the care process) [ 32 , 39 , 40 , 48 ]. Electronic resources supplemented clinic visits through direct communication with people and providers [ 64 ]. Information technology supported the development of ongoing partnerships in innovation and integrating medical and social care to manage chronic illnesses, research, and practice [ 30 , 31 , 32 , 44 , 52 , 78 ].The development integration of technology (e.g., mhealth tools and high-tech and high-touch technology) supported in identifying and engaging high-risk populations [ 53 , 64 , 77 ].

More attention toward changing the organization of the electronic health records system streamlined documentation work of care visits/encounters [ 45 , 56 ]. Improved application, user-focused optimization efforts and tool functionality enabled to address the issues of access, health service and health literacy [ 46 , 59 ]. At the same time, clinicians adopted information technology with the perceived value of data sources enhanced the development of interventions for people living with multimorbidity [ 31 , 48 , 59 ]. Updated electronic health records data analytics incorporated organization-wide procedures (staff, time management, cultivating staff collaborations) and follow-up services in PHC settings [ 39 , 46 , 56 ]. However, coordination and partnership with stakeholders had challenges in healthcare organizations, including work practice discrepancies and lack of enforcement agencies [ 36 , 56 , 76 ]. Additionally, the potential of information challenges influencing PCC was the lack of data protection laws (including documentation and dissemination, time pressure, and conflicting financial incentives) that impeded the use of digital records in care [ 68 ].

Coordination and communication

Prerequisites in co-creating optimal health care practice with and for older people and their expectations influenced the implementation of biomedical and public health interventions and quality of care [ 62 , 66 ]. Coordinated care supports user-driven healthcare decision-making for quality improvement (reducing cost, relationship with providers), a perceived measure of quality care [ 64 , 67 ], common perception, and sustainable primary care models to ensure quality care for physical and emotional health [ 50 , 67 ]. Engaged physician-service users communication found that professionals care (dignity, respect, prioritize, and individualized care) for people with multiple health needs [ 60 , 78 ]. Furthermore, the coordinated care of frontline staff in communicating with other stakeholders can address social and economic issues to implement quality integrated care [ 53 , 63 ], instead of describing the holistic/whole person and PCC approach. GPs’ narrow disease-specific focus of guidelines was inappropriate for addressing people’s needs and health priorities[ 43 ]. Challenges in designing and implementing PCC interventions that hindered the delivery of integrated care were lack of clarity around responsiveness and readiness, lack of information and coordination of care, lack of integrating electronic health records in work practice (preferences, information, and education) [ 41 , 45 , 57 , 68 , 69 ].

Creating an enabling environment

To implement strategies of four categories, it is necessary to create an enabling environment that brings together all stakeholders to undertake transformational change [ 13 ]. Two themes under this component were: flexible management options (17 studies) and enabling environment (17 studies).

Flexible management for care

The flexibility of management can create an enabling environment for PCC. Practice stakeholders address the local needs expectations by redesigning health and social, professional cultures and flexible program implementation [ 37 , 75 ]. Care transitioned from hospital to home toward high-quality care that reduced unnecessary walk-in clinics and emergency department coordinating relationship building (with end users or organizations) and enhanced pharmacy services [ 28 , 36 , 40 , 42 ]. Organizational perspectives (cost-effectiveness and health care delivery processes) improved long-term goal-driven people-centred integrated care and increased people and providers relationships (including knowledge, and satisfaction) [ 32 , 33 , 34 , 47 ]. The PCMH model operationalized health services by providing a feasible reform option and solutions to people's engagement [ 28 , 36 , 44 , 46 ]. However, flexible management and implementation were influenced by challenges (lack of resources and training, excessive caseloads, poor data management responsibilities, lack of medical neighbourhood) and inconsistent implementation of practices [ 36 , 37 , 42 , 46 , 75 ]. Also, difficult communication and being invisible in the context of event-based quality of care frameworks were identified as gaps in primary care clinics [ 41 , 47 ].

Enabling the health system environment

Health workforce attributes (including the responsibility of professionals) enabled sensitizing systems (continuous supervision, professional training, empowerment for leadership) focused quality of care improvement initiatives to bring improved clinical practice [ 69 , 71 , 72 ]. Collaborative works (between a personal network of family and practitioners), upgrading of providers for quality improvement resources, alignment measurement efforts, engaging champions, and need assessment (needs/priorities for people-centred measurement) facilitated identification and management of symptoms [ 72 , 73 , 76 , 77 ]. Similarly, co-location of community health systems, organizations, and service delivery outlets found committed care boundaries that provided sufficient care responsive to their wishes and needs [ 54 , 57 , 75 ]. Mobile health tools are supported ensuring flexible management through sensitization and optimizing the environment across multiple dimensions (individual, provider, and organizational levels) [ 54 , 65 ]. Additionally, understanding common ground, exploring health and illness, valued customers, people-centredness, social and physical wellbeing and satisfaction, whole- PCC reported measures to improve health status and reduce morbidities and mortalities [ 51 , 53 , 58 , 62 ]. Nonetheless, difficulties achieving mutual understanding between end users were influenced by several challenges such as lack of training and new skills of providers, lack of trust (genuine care, respect, dignity, autonomy), poor disclosure of problems (time-compressed visit) and lack of resources [ 60 , 68 , 79 ].

This review synthesizes evidence on people-centred PHC and primary care. Major themes identified from this review were community engagement, empowerment and empathy, leadership and mutual accountability within the organization, home and community-based and participatory care, holistic care for people with multimorbidity, partnership with information technology, coordination and communication, and flexible management for delivery of people-centred PHC services. Most studies in the HICs explained people-centred medical care models with little focus research in LMICs.

There are several ways that health systems could generate and deliver people-centred and integrated care for individuals, families, and communities. Firstly, promoting respectful conversations and activities between care providers and service users is fundamental for improving community empowerment and ensuring providers’ empathy. People engagement and empowerment enhanced people-centred PHC in many contexts. Empowering traditionally disengaged communities and individuals requires awareness of social determinants of health [ 80 ]. Conversation and engagement of people can support personalized, coordinated care towards narrowing inequalities [ 81 ]. The provider’s empathy also enabled supportive, involved care, community, social enterprise, and volunteerism [ 81 ]. Inter-professional teamwork and collaboration with and for older people and relatives are fundamental to empathy and empowerment [ 66 ]. Of the five strategies of the WHO framework on IPCHS, community engagement and empowerment have little attention in the literature. The current global health initiatives, including the Asthana Declaration, have envisioned empowerment, health literacy, and understanding the public’s role in PHC [ 82 ]; community engagement could potentially promote people-centred PHC service delivery. Thus, the focus of research, policy and practices of community engagement and empathy need to be prioritized in PHC and primary care in low-income settings.

Secondly, for PCC and coordinated care, there was an emphasis on organizational integrity and mutual accountability. Strengthening leadership and accountability in home-based care increased people-centred care in PHC services [ 83 ]. Co-creation and healthcare organizations and their leadership efficiently could meet the health needs of people according to standards of care to align tactics and improve organizational reliability while paying attention to quality care [ 84 ]. Organizational leadership and mutual accountability strategies could be beneficial in recruiting people with integrity and sensitivity, the ability to notice and respond through policies of diverse staff and aligning incentives and recognitions [ 11 , 84 ].

Thirdly, some models of care, such as care for people with multiple chronic conditions or comorbidities, residential home-based care, and participatory care, were effective approaches for PCC in PHC and primary care contexts. Such care models can effectively reduce the burden of hospitalization and care costs by using PHC and primary care in prehospital settings [ 83 , 85 ]. The residential home-based model of care facilitates holistic care through collaboration between family members and providers considering the family contexts and comprehensive education and care [ 86 ]. Such a model is useful for people with multiple chronic conditions that could support the activities of daily living and produce high healthcare expenses. Functional limitations can often complicate access to health care, interfere with self‐management, and necessitate reliance on caregivers [ 87 ]. Crucial for implementing people-centred care is knowing and confirming people as a whole and co-creating a tailored personal health plan [ 66 ]. These residential care models could enhance the identification of health priorities (i.e., specific health outcomes and healthcare preferences), and clinicians align their decision-making to achieve these health priorities [ 88 ].

Fourthly, partnership with the digital and information technology sector, and tools can potentially ensure coordinated care by monitoring health records, coordinating processes, tracking health services, and involving people representatives and individuals in developing digital services and work practices. The information technology-related stakeholders are vital for mutual information sharing and distributing initiatives, tasks, and responsibilities from providers to service users [ 89 ]. The human-centred service design approach can leverage the potential of technology and advance healthcare systems, and innovative solutions for healthcare change and wellbeing; addressing the complexity of healthcare systems toward integrated care [ 90 ].

Finally, enabling and flexibly managing the health system environment is fundamental for people-centredness in the provision of delivery of PHC services. System strengthening and management requires system inputs and processes towards desired outcomes. The structural factors of organizations and systems (e.g., creating a PCC culture across the continuum of care, co‐designing educational programs, health promotion and prevention programs with people) provide the foundation for PCC, providing a supportive and accommodating environment developing structures to support health information technology and measure and monitor people-centred care performance influence the processes and outcomes [ 91 ]. The processes component describes the importance of cultivating communication and respectful and compassionate care, engaging service users in managing care and integrating care. At the same time, outcome domains identified include access to care and client-reported outcomes [ 91 ]. At the system level, the enabling environment indicates the adaptation of responses, involvement in support, engagement with professionals, use of information and communication technologies, and organization of care [ 92 ].

This study has some limitations. We included studies written only in English. This study is a scoping review of qualitative evidence in the topic. We synthesized evidence rather than grading the quality of available evidence. Synthesized evidence from this study could provide research, policy, and program insights for improved people-centred PHC services. Evidence generated from this study is primarily based on studies from HICs and upper-middle-income countries (UMICs), which can have limited contextual implications in low-income countries as the health systems contexts of LMICs are different. Therefore, future research can be conducted on specific components of people-centred care in low-income country settings.

Implementing several approaches of people-centred PHC and primary care, especially in HICs, has little priority in LMICs. Potential strategies for PCC could include engaging end users in the care process, community engagement and empowerment, mutual accountability, and institutional leadership. Some successful models of care, such as home-based residential care, are effective in care for people living with multimorbidity, and valuable in prehospital care that can reduce the care costs and burden to the health system. Flexible management options could create an enabling environment for health system strengthening in providing and delivering health services.

Availability of data and materials

All data generated or analyzed during this study are included in this published article [and its supplementary information files].

World Health Organization. People-centered health care: a policy framework. Geneva: World Health Organization; 2014.

Google Scholar  

Goodwin N, Stein V, Amelung V. What is integrated care? Handbook integrated care. 2021. p. 3–25.

Book   Google Scholar  

World Health Organization. The world health report 2008: primary health care now more than ever : introduction and overview. Geneva: World Health Organization; 2008.

Hughes G, Shaw SE, Greenhalgh T. Rethinking Integrated Care: a systematic Hermeneutic Review of the literature on Integrated Care Strategies and concepts. Milbank Q. 2020;98(2):446–92.

Article   PubMed   PubMed Central   Google Scholar  

Hakansson Eklund J, Holmstrom IK, Kumlin T, Kaminsky E, Skoglund K, Hoglander J, Sundler AJ, Conden E, Summer Meranius M. “Same same or different?” A review of reviews of person-centered and patient-centered care. Patient Educ Couns. 2019;102(1):3–11.

Article   PubMed   Google Scholar  

Gusmano MK, Maschke KJ, Solomon MZ. Patient-centered care, yes; patients as consumers, no. Health Aff (Millwood). 2019;38(3):368–73.

Baker L, Bundorf K, Royalty A, Galvin C, McDonald K. Consumer-Oriented strategies for improving health benefit design. 2007.

World Health Organization. People-centred health care: technical papers. In: Tokyo International Forum: International symposium on the people- centred health care: reorienting health systems in the 21st century. 2007.

Ferrer AP, Brentani AV, Sucupira AC, Navega AC, Cerqueira ES, Grisi SJ. The effects of a people-centred model on longitudinality of care and utilization pattern of healthcare services–Brazilian evidence. Health Policy Plan. 2014;29(Suppl 2):ii107-113.

Aberese-Ako M, van Dijk H, Gerrits T, Arhinful DK, Agyepong IA. Your health our concern, our health whose concern?‘: perceptions of injustice in organizational relationships and processes and frontline health worker motivation in Ghana. Health Policy Plan. 2014;29(Suppl 2):ii15-28.

World Health Organization. Continuity and coordination of care: a practice brief to support implementation of the WHO Framework on integrated people-centred health services. Geneva: World Health Organization; 2018.

Sheikh K, Ranson MK, Gilson L. Explorations on people centredness in health systems. In. Oxford: Vol. 26. Oxford University Press; 2014. p. ii1–5.

World Health Organization. Framework on integrated, people-centred health services. Geneva: World Health Organization; 2016.

World Health Organization. Framework on integrated people-centred health services: an overview. Geneva: World Health Organization; 2016.

World Health Organization. A vision for primary health care in the 21st century: towards universal health coverage and the sustainable development goals. Geneva: World Health Organization and the United Nations Children’s Fund (UNICEF); 2020.

Huffstetler AN, Phillips Jr RL, Leyns CC, Willis JS, Canchihuaman FA. People-centered health services. In: Person centered medicine. Cham: Springer; 2023: p. 135–50.

Peters MDJ, Marnie C, Colquhoun H, Garritty CM, Hempel S, Horsley T, Langlois EV, Lillie E, O’Brien KK, Tunçalp Ö, et al. Scoping reviews: reinforcing and advancing the methodology and application. Syst Rev. 2021;10(1):263.

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):69.

McGowan J, Straus S, Moher D, Langlois EV, O’Brien KK, Horsley T, Aldcroft A, Zarin W, Garitty CM, Hempel S, et al. Reporting scoping reviews-PRISMA ScR extension. J Clin Epidemiol. 2020;123:177–9.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, et al. PRISMA Extension for scoping reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467–73.

Briggs J. Systematic review resource package. The Joanna Briggs Institute Method for systematic review research quick reference guide version. 2015. p. 4.

Munn Z, Peters MD, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):1–7.

Article   Google Scholar  

Peterson K, Anderson J, Bourne D, Charns MP, Gorin SS, Hynes DM, McDonald KM, Singer SJ, Yano EM. Health Care Coordination theoretical frameworks: a systematic scoping review to increase their understanding and use in practice. J Gen Intern Med. 2019;34(Suppl 1):90–8.

Pham MT, Rajic A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5(4):371–85.

Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10(Suppl 11suppl):21–34.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13(1):1–8.

Scott MA, Hitch B, Ray L, Colvin G. Integration of pharmacists into a patient-centered medical home. J Am Pharm Assoc (2003). 2011;51(2):161–6.

Council LS, Geffken D, Valeras AB, Orzano AJ, Rechisky A, Anderson S. A medical home: changing the way patients and teams relate through patient-centered care plans. Fam Syst Health. 2012;30(3):190–8.

Hudon C, Fortin M, Haggerty J, Loignon C, Lambert M, Poitras ME. Patient-centered care in chronic Disease management: a thematic analysis of the literature in family medicine. Patient Educ Couns. 2012;88(2):170–6.

Leventhal T, Taliaferro JP, Wong K, Hughes C, Mun S. The patient-centered medical home and health information technology. Telemed J E Health. 2012;18(2):145–9.

Reed P, Conrad DA, Hernandez SE, Watts C, Marcus-Smith M. Innovation in patient-centered care: lessons from a qualitative study of innovative health care organizations in Washington State. BMC Fam Pract. 2012;13: 120.

Van Berckelaer A, DiRocco D, Ferguson M, Gray P, Marcus N, Day S. Building a patient-centered medical home: obtaining the patient’s voice. J Am Board Fam Med. 2012;25(2):192–8.

Lavoie JG, Wong ST, Chongo M, Browne AJ, MacLeod ML, Ulrich C. Group medical visits can deliver on patient-centred care objectives: results from a qualitative study. BMC Health Serv Res. 2013;13:155.

Rosstad T, Garasen H, Steinsbekk A, Sletvold O, Grimsmo A. Development of a patient-centred care pathway across healthcare providers: a qualitative study. BMC Health Serv Res. 2013;13:121.

Akinci F, Patel PM. Quality improvement in healthcare delivery utilizing the patient-centered medical home model. Hosp Top. 2014;92(4):96–104.

Grace SM, Rich J, Chin W, Rodriguez HP. Flexible implementation and integration of new team members to support patient-centered care. Healthc (Amst). 2014;2(2):145–51.

Mead H, Andres E, Regenstein M. Underserved patients’ perspectives on patient-centered primary care: does the patient-centered medical home model meet their needs? Med Care Res Rev. 2014;71(1):61–84.

Flink M, Bergenbrant Glas S, Airosa F, Ohlen G, Barach P, Hansagi H, Brommels M, Olsson M. Patient-centered handovers between hospital and primary health care: an assessment of medical records. Int J Med Inform. 2015;84(5):355–62.

Brown JB, Ryan BL, Thorpe C. Processes of patient-centred care in Family Health teams: a qualitative study. CMAJ Open. 2016;4(2):E271-276.

Doubova SV, Guanais FC, Perez-Cuevas R, Canning D, Macinko J, Reich MR. Attributes of patient-centered primary care associated with the public perception of good healthcare quality in Brazil, Colombia, Mexico and El Salvador. Health Policy Plan. 2016;31(7):834–43.

Friedman A, Howard J, Shaw EK, Cohen DJ, Shahidi L, Ferrante JM. Facilitators and barriers to care coordination in patient-centered Medical homes (PCMHs) from coordinators’ perspectives. J Am Board Fam Med. 2016;29(1):90–101.

Hansen E, Walters J, Howes F. Whole person care, patient-centred care and clinical practice guidelines in general practice. Health Sociol Rev. 2016;25(2):157–70.

Kogan AC, Wilber K, Mosqueda L. Person-centered care for older adults with chronic conditions and functional impairment: a systematic literature review. J Am Geriatr Soc. 2016;64(1):e1-7.

Zhang J, Chen Y, Ashfaq S, Bell K, Calvitti A, Farber NJ, Gabuzda MT, Gray B, Liu L, Rick S, et al. Strategizing EHR use to achieve patient-centered care in exam rooms: a qualitative study on primary care providers. J Am Med Inform Assoc. 2016;23(1):137–43.

Agha AZ, Werner RM, Keddem S, Huseman TL, Long JA, Shea JA. Improving patient-centered care: how clinical staff overcome barriers to Patient Engagement at the VHA. Med Care. 2018;56(12):1009–17.

Berntsen G, Hoyem A, Lettrem I, Ruland C, Rumpsfeld M, Gammon D. A person-centered integrated care quality framework, based on a qualitative study of patients’ evaluation of care in light of chronic care ideals. BMC Health Serv Res. 2018;18(1):479.

Poitras ME, Maltais ME, Bestard-Denomme L, Stewart M, Fortin M. What are the effective elements in patient-centered and multimorbidity care? A scoping review. BMC Health Serv Res. 2018;18(1):446.

Reeve J. Primary care redesign for person-centred care: delivering an international generalist revolution. Aust J Prim Health. 2018;24(4):330–6.

Kamimura A, Weaver S, Armenta B, Gull B, Ashby J. Patient centeredness: the perspectives of uninsured primary care patients in the United States. Int J Care Coord. 2019;22(1):19–26.

Kuipers SJ, Cramm JM, Nieboer AP. The importance of patient-centered care and co-creation of care for satisfaction with care and physical and social well-being of patients with multi-morbidity in the primary care setting. BMC Health Serv Res. 2019;19(1):13.

Lionis C, Symvoulakis EK, Markaki A, Petelos E, Papadakis S, Sifaki-Pistolla D, Papadakakis M, Souliotis K, Tziraki C. Integrated people-centred primary health care in Greece: unravelling Ariadne’s thread. Prim Health Care Res Dev. 2019;20:e113.

Millenson ML, Muhlestein DB, O’Donnell EM, Northam Jones DA, Haring RS, Merrill T, Weissman JS. Patient-centered care innovations by accountable care organizations: Lessons from leaders. Healthc (Amst). 2019;7(4):1–5.

Petit-Steeghs V, Lips SR, Schuitmaker-Warnaar TJ, Broerse JEW. Client-centred maternity care from women’s perspectives: need for responsiveness. Midwifery. 2019;74:76–83.

Santana MJ, Ahmed S, Lorenzetti D, Jolley RJ, Manalili K, Zelinsky S, Quan H, Lu M. Measuring patient-centred system performance: a scoping review of patient-centred care quality indicators. BMJ Open. 2019;9(1): e023596.

Schimmer R, Orre C, Oberg U, Danielsson K, Hornsten A. Digital person-centered self-management support for people with type 2 Diabetes: qualitative study Exploring Design challenges. JMIR Diabetes. 2019;4(3):e10702.

Waweru E, Sarkar NDP, Ssengooba F, Gruenais ME, Broerse J, Criel B. Stakeholder perceptions on patient-centered care at primary health care level in rural eastern Uganda: a qualitative inquiry. PLoS ONE. 2019;14(8):e0221649.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Brickley B, Sladdin I, Williams LT, Morgan M, Ross A, Trigger K, Ball L. A new model of patient-centred care for general practitioners: results of an integrative review. Fam Pract. 2020;37(2):154–72.

PubMed   Google Scholar  

Holt JM, Cusatis R, Asan O, Williams J, Nukuna S, Flynn KE, Moore J, Crotty BH. Incorporating patient-generated contextual data into care: clinician perspectives using the Consolidated Framework for implementation science. Healthc (Amst). 2020;8(1):100369.

Kuipers SJ, Nieboer AP, Cramm JM. Views of patients with multi-morbidity on what is important for patient-centered care in the primary care setting. BMC Fam Pract. 2020;21(1):71.

Turner RE, Archer E. Patient-centred care: the patients’ perspective - A mixed-methods pilot study. Afr J Prim Health Care Fam Med. 2020;12(1):e1–8.

Waweru E, Smekens T, Orne-Gliemann J, Ssengooba F, Broerse J, Criel B. Patient perspectives on interpersonal aspects of healthcare and patient-centeredness at primary health facilities: a mixed methods study in rural Eastern Uganda. PLoS ONE. 2020;15(7):e0236524.

Ahmed A, van den Muijsenbergh M, Mewes JC, Wodchis WP, Vrijhoef HJM. Untangling the inter-relatedness within integrated care programmes for community-dwelling frail older people: a rapid realist review. BMJ Open. 2021;11(4):e043280.

Brault MA, Curry LA, Kershaw TS, Singh K, Vash-Margita A, Camenga DR. Supporting patient-centered communication on adolescent sexual and Reproductive Health-perspectives to build an Appointment Planning Tool. J Pediatr Adolesc Gynecol. 2021;34(5):725–31.

Brickley B, Williams LT, Morgan M, Ross A, Trigger K, Ball L. Putting patients first: development of a patient advocate and general practitioner-informed model of patient-centred care. BMC Health Serv Res. 2021;21(1):261.

Ebrahimi Z, Patel H, Wijk H, Ekman I, Olaya-Contreras P. A systematic review on implementation of person-centered care interventions for older people in out-of-hospital settings. Geriatr Nurs. 2021;42(1):213–24.

Epstein DS, Barton C, Prathivadi P, Mazza D. Patient-chosen gap payment: an exploratory qualitative review of patients and general practitioner attitudes toward an alternative funding model for general practice. Aust J Prim Health. 2021;27(4):259–64.

Kuipers SJ, Nieboer AP, Cramm JM. Easier Said Than Done: Healthcare professionals’ barriers to the provision of patient-centered primary care to patients with multimorbidity. Int J Environ Res Public Health. 2021;18(11):6057.

Kumar M, Osborn TL, Mugo C, Akbarialiabad H, Warfa O, Mbuthia WM, Wambugu C, Ngunu C, Gohar F, Mwaniga S, et al. A four-component Framework toward Patient-Centered, Integrated Mental Healthcare in Kenya. Front Public Health. 2021;9: 756861.

Raj M, Platt JE, Anthony D, Fitzgerald JT, Lee SD. What does patient-centered Mean? Qualitative perspectives from older adults and family caregivers. Gerontol Geriatr Med. 2021;7:23337214211017610.

Westlake D, Ekman I, Britten N, Lloyd H. Terms of engagement for working with patients in a person-centred partnership: a secondary analysis of qualitative data. Health Soc Care Community. 2022;30(1):330–40.

Armenta-Arellano S, Munos-Hernandez JA, Pavon-Leon P, Coronel-Brizio PG, Gutierrez-Alba G. An overview on the promotion on patient-centered care and shared decision-making in Mexico. Z Evid Fortbild Qual Gesundhwes. 2022;171:93–7.

Aworinde J, Ellis-Smith C, Gillam J, Roche M, Coombes L, Yorganci E, Evans CJ. How do person-centered outcome measures enable shared decision-making for people with Dementia and family carers?-A systematic review. Alzheimers Dement (N Y). 2022;8(1):e12304.

Kari H, Kortejarvi H, Laaksonen R. Developing an interprofessional people-centred care model for home-living older people with multimorbidities in a primary care health centre: a community-based study. Explor Res Clin Soc Pharm. 2022;5:100114.

PubMed   PubMed Central   Google Scholar  

Lalani M, Marshall M. Co-location, an enabler for service integration? Lessons from an evaluation of integrated community care teams in East London. Health Soc Care Community. 2022;30(2):e388-396.

Lateef MA, Mhlongo EM. A qualitative study on patient-centered care and perceptions of nurses regarding primary healthcare facilities in Nigeria. Cost Eff Resour Alloc. 2022;20(1):40.

Manalili K, Scott CM, Hemmelgarn B, O’Beirne M, Bailey AL, Haener MK, Banerjee C, Peters SP, Chiodo M, Aghajafari F, et al. Co-designing person-centred quality indicator implementation for primary care in Alberta: a consensus study. Res Involv Engagem. 2022;8(1):59.

Schuttner L, Hockett Sherlock S, Simons CE, Johnson NL, Wirtz E, Ralston JD, Rosland AM, Nelson K, Sayre G. My goals are not their goals: barriers and facilitators to delivery of patient-centered care for patients with Multimorbidity. J Gen Intern Med. 2022;37(16):4189–96.

Tuepker A, Newell S, Sorrentino A, Cusack M, True G, Dichter ME. High-risk encounters: primary care experiences of women living with intimate Partner Violence, and implications for the patient centered Medical Home. J Aggress Maltreatment Trauma. 2022;32(7–8):1041–54.

Acha BV, Ferrandis ED, Ferri Sanz M, Garcia MF. Engaging people and co-producing research with persons and communities to foster person-centred care: a meta-synthesis. Int J Environ Res Public Health. 2021;18(23):12334.

World Health Organization. WHO community engagement framework for quality, people-centred and resilient health services. Geneva: World Health Organization; 2017.

Jungo KT, Anker D, Wildisen L. Astana declaration: a new pathway for primary health care. Int J Public Health. 2020;65(5):511–2.

World Health Organization. People-centred and integrated health services: an overview of the evidence: interim report. Geneva: World Health Organization; 2015.

Allwood D, Koka S, Armbruster R, Montori V. Leadership for careful and kind care. BMJ Lead. 2022;6(2):125–9.

World Health Organization. Rebuilding for sustainability and resilience: strengthening the integrated delivery of long-term care in the European Region. Geneva: World Health Organization. Regional Office for Europe; 2022.

Kokorelias KM, Gignac MAM, Naglie G, Cameron JI. Towards a universal model of family centered care: a scoping review. BMC Health Serv Res. 2019;19(1):564.

Health UDO, Services H. Multiple chronic conditions—a strategic framework: optimum health and quality of life for individuals with multiple chronic conditions. Washington: US Department of Health and Human Services; 2010. p. 2.

Tinetti ME, Naik AD, Dindo L, Costello DM, Esterson J, Geda M, Rosen J, Hernandez-Bigos K, Smith CD, Ouellet GM, et al. Association of Patient priorities-aligned decision-making with patient outcomes and Ambulatory Health Care Burden among older adults with multiple chronic conditions: a Nonrandomized Clinical Trial. JAMA Intern Med. 2019;179(12):1688–97.

Granstrom E, Wannheden C, Brommels M, Hvitfeldt H, Nystrom ME. Digital tools as promoters for person-centered care practices in chronic care? Healthcare professionals’ experiences from rheumatology care. BMC Health Serv Res. 2020;20(1):1108.

Patrício L, Sangiorgi D, Mahr D, Čaić M, Kalantari S, Sundar S. Leveraging service design for healthcare transformation: toward people-centered, integrated, and technology-enabled healthcare systems. J Service Manage. 2020;31(5):889–909.

Santana MJ, Manalili K, Jolley RJ, Zelinsky S, Quan H, Lu M. How to practice person-centred care: a conceptual framework. Health Expect. 2018;21(2):429–40.

Loizeau V, Morvillers JM, Bertrand DP, Kilpatrick K, Rothan-Tondeur M. Defining an enabling environment for those with chronic Disease: an integrative review. BMC Nurs. 2021;20(1):252.

Download references

Acknowledgements

Not applicable.

No funding was received for the development of this manuscript.

Author information

Authors and affiliations.

School of Public Health, Faculty of Medicine, the University of Queensland, Brisbane, QLD, Australia

Resham B. Khatri, Aklilu Endalamaw & Yibeltal Assefa

Health Social Science and Development Research Institute, Kathmandu, Nepal

Resham B. Khatri

International Institute for Primary Health Care-Ethiopia, Addis Ababa, Ethiopia

Eskinder Wolka, Frehiwot Nigatu & Anteneh Zewdie

Centre for Applied Health Economics, School of Medicine, Griffith University, Southport, QLD, Australia

Daniel Erku

Menzies Health Institute Queensland, Griffith University, Southport, QLD, Australia

College of Medicine and Health Sciences, Bahir Dar University, Bahir Dar, Ethiopia

Aklilu Endalamaw

You can also search for this author in PubMed   Google Scholar

Contributions

RBK: Data collection, analysis, and preparation of the first draft of the manuscript. RBK, AE, DE, and YA: conceived the study and interpreted the findings. YA: Supervision of the study. RBK, DE, AE, EW, FN, AZ, and YA: provided critical comments in the revision of the manuscript. All authors agreed and approved the final version of the manuscript.

Corresponding author

Correspondence to Resham B. Khatri .

Ethics declarations

Ethics approval and consent to participate.

This paper synthesizes evidence from other published studies; thus, ethics approval was not required. No new datasets were generated or analyzed for this study that require ethics approval and participants' consent.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: table s1..

Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist. Table S2. A summary of studies included in the review.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Khatri, R.B., Wolka, E., Nigatu, F. et al. People-centred primary health care: a scoping review. BMC Prim. Care 24 , 236 (2023). https://doi.org/10.1186/s12875-023-02194-3

Download citation

Received : 28 July 2023

Accepted : 27 October 2023

Published : 09 November 2023

DOI : https://doi.org/10.1186/s12875-023-02194-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• People-centred care
• Integrated care
• Primary health care
• Primary care

BMC Primary Care

ISSN: 2731-4553

• General enquiries: [email protected]

• Methodology
• Open access
• Published: 06 April 2024

Consolidated guidance for behavioral intervention pilot and feasibility studies

• Christopher D. Pfledderer   ORCID: orcid.org/0000-0001-7503-8554 1 , 2 ,
• Lauren von Klinggraeff 3 ,
• Sarah Burkart 3 ,
• Alexsandra da Silva Bandeira 3 ,
• David R. Lubans 4 ,
• Russell Jago 5 ,
• Anthony D. Okely 6 ,
• Esther M. F. van Sluijs 7 ,
• John P. A. Ioannidis 11 , 12 , 8 , 9 , 10 ,
• James F. Thrasher 3 ,
• Xiaoming Li 3 &
• Michael W. Beets 3  

Pilot and Feasibility Studies volume  10 , Article number:  57 ( 2024 ) Cite this article

254 Accesses

20 Altmetric

Metrics details

In the behavioral sciences, conducting pilot and/or feasibility studies (PFS) is a key step that provides essential information used to inform the design, conduct, and implementation of a larger-scale trial. There are more than 160 published guidelines, reporting checklists, frameworks, and recommendations related to PFS. All of these publications offer some form of guidance on PFS, but many focus on one or a few topics. This makes it difficult for researchers wanting to gain a broader understanding of all the relevant and important aspects of PFS and requires them to seek out multiple sources of information, which increases the risk of missing key considerations to incorporate into their PFS. The purpose of this study was to develop a consolidated set of considerations for the design, conduct, implementation, and reporting of PFS for interventions conducted in the behavioral sciences.

To develop this consolidation, we undertook a review of the published guidance on PFS in combination with expert consensus (via a Delphi study) from the authors who wrote such guidance to inform the identified considerations. A total of 161 PFS-related guidelines, checklists, frameworks, and recommendations were identified via a review of recently published behavioral intervention PFS and backward/forward citation tracking of a well-known PFS literature (e.g., CONSORT Ext. for PFS). Authors of all 161 PFS publications were invited to complete a three-round Delphi survey, which was used to guide the creation of a consolidated list of considerations to guide the design, conduct, and reporting of PFS conducted by researchers in the behavioral sciences.

A total of 496 authors were invited to take part in the three-round Delphi survey (round 1, N  = 46; round 2, N  = 24; round 3, N  = 22). A set of twenty considerations, broadly categorized into six themes (intervention design, study design, conduct of trial, implementation of intervention, statistical analysis, and reporting) were generated from a review of the 161 PFS-related publications as well as a synthesis of feedback from the three-round Delphi process. These 20 considerations are presented alongside a supporting narrative for each consideration as well as a crosswalk of all 161 publications aligned with each consideration for further reading.

We leveraged expert opinion from researchers who have published PFS-related guidelines, checklists, frameworks, and recommendations on a wide range of topics and distilled this knowledge into a valuable and universal resource for researchers conducting PFS. Researchers may use these considerations alongside the previously published literature to guide decisions about all aspects of PFS, with the hope of creating and disseminating interventions with broad public health impact.

Peer Review reports

Key messages regarding feasibility

• There are more than 160 published guidelines, reporting checklists, frameworks, and recommendations related to PFS. All these publications offer some form of guidance on PFS, but many focus on one or a few topics, making it difficult for researchers wanting to gain a broader understanding of all the relevant and important aspects of PFS and requires them to seek out multiple sources of information, which increases the risk of missing key considerations to incorporate into their PFS.

• We present a set of consolidated considerations for behavioral intervention pilot and/or feasibility studies based on a review of the literature and a Delphi study with the authors who wrote this literature.

• We believe this consolidated set of considerations can be a “go-to” resource for any behavioral interventionist wanting to design, conduct, and report on their pilot and/or feasibility study.

In the behavioral sciences, conducting pilot and/or feasibility studies (PFS) is a key step that occurs early in the translational science continuum. PFS provide essential information to inform the design, conduct, and implementation of larger-scale trials, although not all studies follow the traditional roadmap to scale-up [ 1 ]. PFS are designed to answer questions surrounding uncertainty (feasibility) and potential impact (preliminary efficacy) and to inform gaps in knowledge about the various aspects of the intervention or conduct of the study. In turn, this information is used to make decisions regarding scale-up and future plans for a larger-scale trial.

There are more than 160 published guidelines, checklists, frameworks, and recommendations related to the design, conduct, and reporting of PFS. These publications offer some form of guidance on PFS, but many focus on a specific aspect of design, conduct, and reporting considerations. This makes it difficult for researchers who want to gain a broader understanding of all the relevant and important aspects of PFS and forces them to seek out multiple sources of information, which increases the risk of missing key considerations to incorporate into their PFS. Because of this, we believe a consolidated list of considerations, drawing on the breadth and depth of knowledge that has already been published on the topic, would have high utility for researchers and assist them in understanding important considerations and nuances when conducting a PFS.

Throughout this paper, we refer to PFS as early-stage studies designed to inform larger-scale, well-powered trials. We recognize that there are numerous labels for such studies (e.g., “proof-of-concept”, “evidentiary”, “vanguard”). We also realize that the terms “pilot” and “feasibility” evoke different meanings [ 2 , 3 ] and are used interchangeably and, in some instances, simultaneously. We address this issue in this consolidation of considerations. We also recognize that not all PFS will include or need to consider all the identified considerations. In many instances, however, a single PFS is designed to cover all of the data needed to inform a larger-scale trial [ 4 ]. This includes everything from estimating recruitment/retention rates, participant satisfaction and engagement, fidelity, and a host of other feasibility indicators, as well as providing some preliminary indications of change in one or more outcomes of interest. Researchers often deliberately design a PFS to collect information across these multiple dimensions, though their decision making is often largely driven by such issues as available resources and abbreviated timelines.

The purpose of this study was to develop a consolidated set of considerations for the design, conduct, implementation, and reporting of PFS for interventions in the fields of behavioral sciences. The considerations presented herein were developed through any extensive review of the literature and a Delphi study of experts who wrote the existing literature on PFS. The consolidated set of considerations was developed for universal application across interventions in the behavioral sciences and across the study designs one may choose. We expect this consolidation will serve as a valuable resource for all behavioral science interventionists who design and conduct PFS, regardless of the intervention mechanism, target population, or study design.

To ensure rigor and methodological quality throughout the consolidation of previously published guidelines, checklists, frameworks, and recommendations, we relied on guidance from Moher et al., [ 5 , 6 ] which details the main steps in the development of evidence-based consensus in health fields. These steps included developing a strong rationale for the consolidation, necessary preparatory work conducted by the study team, consensus activities, and development of the final consolidation. These steps are detailed below. When relevant, we also drew on similar consensus studies conducted in the behavioral sciences [ 2 , 3 , 7 , 8 ].

Review of previously published guidelines, checklists, frameworks, and recommendations for PFS

A scoping bibliometric review of published PFS-related guidelines, checklists, frameworks, and recommendations was conducted prior to developing the Delphi survey, which has been reported elsewhere [ 9 ]. Briefly, we identified 4143 PFS from which we then identified 90 guidelines, checklists, frameworks, and recommendations cited in that literature. We then continued searching for relevant literature via backward citation tracking of known publications, including the CONSORT Extension for Pilot and Feasibility Studies [ 7 ], Medical Research Council guidance [ 10 ], and publications such as Bowen et al. [ 11 ] and Pearson et al. [ 12 ] A total of 161 publications were identified that encompassed nine thematic domains: adaptations , definitions of pilot and feasibility studies , design and interpretation , feasibility , implementation , intervention development , progression criteria , sample size , and scale-up . The 161 publications guided our inclusion of the sample of respondents for the Delphi survey, which is detailed in the next section. It is worth noting that after this review, we identified an additional relevant publication published after the completion of the study, which is included in our final sample (bringing the total number of studies to 162) but was not used to inform the Delphi study.

Participant selection and recruitment for the Delphi survey

Lead, second, corresponding, and senior authors of the 161 published guidelines, checklists, frameworks, and recommendations for PFS were invited via email to complete a three-round Delphi study. Contact information was retrieved from published article meta-data and when not found in the published articles, emails were retrieved from another publicly available source, such as faculty pages or university websites. This resulted in 496 potential participants, who were sent an individualized invitation email via Qualtrics for round 1 of the Delphi study. For round 2, only participants who completed round 1 were invited to take part in the survey. We then sent the round 3 survey back to the original pool of 496 potential participants, regardless of whether they completed round 1. This process is summarized in Fig.  1 and took place between May 2022 and January 2023. Ethical approval was granted by the University of South Carolina’s Institutional Review Board (IRB # Pro00120890) prior to the start of the study.

Participant flow through each round of the Delphi survey process

Delphi survey

Each round of the Delphi survey process was guided by established protocols [ 13 , 14 ] and is detailed below.

Round 1—Delphi survey

In round 1 of the Delphi process, participants were asked to provide the most important considerations regarding the design, conduct, analysis, or reporting of behavioral pilot and/or feasibility intervention studies in separate free-text fields via Qualtrics. Before beginning the survey, participants were provided with operational definitions of both “behavioral interventions” and “preliminary studies” for context. No other prompts were provided. In round 1 of the Delphi study, we referred to PFS as “preliminary” studies, but after receiving comments about the use of this term, this was changed to “pilot and/or feasibility” studies in round 2. Survey distribution for round 1 took place in May and June 2022.

Preparation for round 2

Participants’ responses from round 1 were exported from Qualtrics to a.csv file in Microsoft Excel, collated into individual Microsoft Word documents for each participant, converted to PDFs, and imported into NVivo for thematic coding. Prior to coding responses in NVivo, we simplified and revised our original nine thematic domains from the scoping bibliometric review into six overarching themes: intervention design , study design , conduct of trial , implementation of intervention , statistical analysis , and reporting . This revision was conducted after an initial review of responses from round 1 of the Delphi survey in an effort to simplify themes and to allow for maximum parsimony across expert perspectives. Specifically, we identified overlap in several of the original nine themes and made a decision to include them as subthemes in the revision to six overarching themes. The titles of the original nine thematic domains were largely retained and can be found embedded as subthemes in the six revised overarching themes. A two-step thematic coding process followed. First, individual participant responses were coded into a corresponding theme based on the content of their response. This was completed by two members of the research team (CDP and MWB). Disagreements were brought to the larger research team (LV, SB, and AB) during weekly meetings and were resolved at that time. Once participant responses were coded into one of the six overarching themes, our research team coded responses into one of 20 subthemes based on qualitative analysis of participants’ responses by theme. These 20 subthemes served as the coding framework for the second step of the thematic coding process, and responses were coded as such by two members of the research team (CDP and MWB).

Round 2—Delphi survey

In round 2 of the Delphi study, participants were re-oriented to the study with a brief narrative and were presented with the six overarching themes and 20 subthemes generated via qualitative analysis of the results from round 1. To give participants context, we provided select, representative quotes for each subtheme from round 1 of the survey. After being presented with the theme, subtheme, and select quotes, participants were asked to provide a recommendation for each subtheme for inclusion in a consolidated framework for behavioral intervention PFS. Participants were also given the chance to indicate if they felt a subtheme should not be included in a consolidated framework. The survey was organized such that each theme (along with the corresponding subthemes) was presented as a randomized block, meaning individual participants were presented with a unique order of themes and asked to provide their considerations. Block randomization of themes was performed to prevent the possibility of homogenous burnout across participants as they reached the last theme of the survey. The last question of the survey was a free-text field in which participants could indicate if there were any additional considerations that were not mentioned in the survey that should be added to a consolidated framework for pilot and/or feasibility behavioral intervention studies. Survey distribution for round 2 took place in September and October 2022.

Preparation for round 3

Participant responses from round 2 were exported from Qualtrics to a.csv file in Microsoft Excel and collated into individual Microsoft Word documents for each of the 20 subthemes. A collection of considerations for each subtheme was written based on participant responses from rounds 1 and 2 and from information provided throughout the previously identified 161 pilot and/or feasibility-related guidelines, checklists, frameworks, and recommendations. Weekly research group meetings were used to further refine the considerations.

Round 3—Delphi survey

In the final round of the Delphi study, participants were first asked to provide basic demographic information including age, sex, race/ethnicity, and the year in which they received their terminal degree. Demographic information was not collected from participants in round 1 or 2 of the Delphi survey to limit participant burden in the initial rounds of the survey. We then provided participants with an outline of the six themes and 20 subthemes that emerged from rounds 1 and 2 of the study, a description of the final recommendation for the study, and instructions for the final survey. For each of the 20 subthemes, participants were given an operational definition of the subtheme and a list of considerations, which were generated based on the comments from rounds 1 and 2. They were then asked to rate their level of agreement with the considerations (0–10 Likert scale from Strongly Disagree to Strongly Agree). An optional free-text field was provided for additional information about what we should add to/change about the considerations. Participants were presented with each subtheme in block-randomized order just as in round 2. Survey distribution for round 3 took place in December 2022 and January 2023.

Final consolidation of considerations

The final set of considerations was written in a similar manner to round 2. Responses were collated into separate working documents for each of the 20 subthemes, which also included the list of previously written considerations drafted for round 2. The previously written considerations were altered based on participant feedback from round 3 and from further supporting information from the 161 pilot and/or feasibility-related guidelines, checklists, frameworks, and recommendations. Primary changes to the considerations were made by two members of the research team (CDP and MWB) and further refined by members of our larger research team (LV, SB, and AB).

Analysis of quantitative data

There were two forms of quantitative data gathered from participants during round 3 of the Delphi survey process. The first was demographic information, which was summarized descriptively as means, standard deviations, and ranges where appropriate. The second were the participant’s Likert-scale ratings of each set of considerations for each of the 20 subthemes. These data were summarized visually with boxplots and descriptively with means, standard deviations, medians, ranges, and interquartile ranges. All quantitative analysis was performed in STATA v17.0 statistical software package (College Station, TX, USA).

Participant characteristics and survey completion

A total of 46 of the 496 (9.3%) invited authors representing 51 of the 161 (31.7%) identified publications completed round 1 of the Delphi study. In round 1, where respondents were asked to provide up to 20 considerations regarding the design, conduct, analysis, or reporting of behavioral pilot and/or feasibility intervention studies, participants gave a mean of 8 ± 4 (range = 1–20, median = 7, IQR = 5–10) considerations. Of the 46 participants who completed round 1, 24 (52.2%) completed round 2. A total of 50 (10.1%) of the original pool of 496 participants representing 60 (37.3%) publications completed round 3. For the 161 publications that were represented by authors in the Delphi study, the median year of publication was 2015 (range = 1998–2022, IQR = 2013–2018). Comparatively, across all possible 161 identified publications, the median year of publication was 2013 (range = 1989–2022, IQR = 2009–2017). A visual summary of participant flow through each of the three rounds of the Delphi survey process is provided in Fig.  1 . Demographic information for participants who completed round 3 is presented in Table  1 .

Likert ratings of the considerations

Likert scale ratings (0–10 scale) of each of the considerations for the 20 subthemes were provided by 50 out of 50 (100%) participants during round 3 of the Delphi survey. These are summarized in Table  2 . Average ratings for considerations across all 20 subthemes ranged from 7.6 to 8.8, with medians ranging from 8 to 10.

Consolidated considerations for PFS

For each subtheme, we provide an operational definition of the subtheme, a consolidated list of considerations based on the review of pilot and/or feasibility literature and the three-round Delphi study, and a narrative summary of the subtheme. We also provide a crosswalk of 161 guidelines, checklists, frameworks, and recommendations, mapped on to the subthemes identified and an additional publication that was published after the Delphi process, but was relevant to include in the list [ 15 ]. The crosswalk is found in Additional file 1 and can be used to identify supporting literature for each of the subthemes and considerations we have consolidated. Of the 161 publications, 15 are reporting guidelines/checklists, 44 are guidelines/recommendations, 18 are reviews that offer recommendations, 37 are frameworks/models, and 47 are commentaries/editorials that offer recommendations or guidance for preliminary studies. For the narrative summary, wherever possible, we have identified relevant examples across widely used study designs for PFS which range from “N of 1” studies, micro-randomized trials, single and multiple group designs, and those involving traditional randomization, to highlight the universality of the consolidated considerations.

Intervention design

Adaptations and tailoring.

Adaptations and tailoring refer to any deliberate changes to the design or delivery of an intervention, with the goal of improving fit or effectiveness in a given context [ 16 ].

Considerations

Where components of the intervention are adapted/tailored, details of who was involved (e.g., investigative team, key stakeholders, participants) in the decisions (see 1.3. Stakeholder Engagement and Co-Production ), when the adaptations/tailoring occurred, and how and why the modification(s) were made need to be clearly reported.

How the proposed adaptations/tailoring address the issues/challenges observed in the intervention need to be clearly reported along with justification for why these changes should result in an improved design.

Whether the adaptations/tailoring occurred a priori or during the conduct of the study should be clearly described.

The intervention component of PFS can be conducted in a rigorous fashion yet be flexible enough to allow for minor adaptations or tailoring (in composition, format, design, etc.) when justified and in response to emerging feasibility indicators.

If substantial adaptations are made to the intervention, such that the adaptations may influence feasibility indicators or behavioral outcomes, re-testing of the PFS prior to progression is justifiable (see 2.1. Iteration and Intervention Refinement ). Adaptations/tailoring occurring under these circumstances should refer to any a priori progression criteria specifications (see 2.2. Progression Criteria ).

Often, existing evidence-based interventions are modified (i.e., adapted/tailored) for delivery to a new sample or in a new setting that is different from where the intervention was originally implemented and evaluated. In these situations, a PFS may be conducted to establish whether the modifications are appropriate in the new sample/setting [ 17 , 18 ]. Adaptations are often made to increase relevance and participant engagement, with the assumption the adaptations would lead to better outcomes in the target populations and settings of eventual interest [ 19 , 20 ].

Adaptations can consist of changes to intervention materials to make them culturally relevant to the target population (race/ethnicity, country/setting, norms/values) [ 19 , 21 ]. Adaptations may also include changes to the intervention itself, such as how it is delivered (e.g., combining sessions, online vs. face-to-face), delivery location, who it is delivered by, or the length of the sessions/intervention [ 22 , 23 ]. Adaptations may occur at any point in the design, implementation, and evaluation/interpretation of a PFS. These include a priori adaptations of existing interventions, those that occur as a result of the evaluation of an intervention, or adaptations made on an ongoing basis throughout a PFS [ 19 , 21 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 ].

Where adaptations/tailoring occur, reasons for the adaptations and who participated in the decision-making process should be reported. Often, the adaptation process includes coproduction/codesign methods that can involve focus groups, feedback sessions, and key patient, participant, and public involvement [ 17 ] to justify and inform the relevancy of the adaptations [ 19 , 34 , 35 , 36 ] (see 1.3. Stakeholder Engagement and Co-Production). If coproduction/codesign methods are used, these should be clearly reported.

Site selection and context

Site selection refers to the location in which a PFS will be conducted. Context refers to the factors that form the setting of the intervention, including location, culture, environment, and situation [ 12 , 37 ].

Whenever feasible, researchers should choose sites for PFS that are representative of those anticipated in the future larger-scale trial.

Purposeful selection of sites can be used to ensure an intervention is tested in an appropriate range of contexts.

A rationale for the sites selected should be clearly stated along with how the sites and context reflect what is anticipated in the future larger-scale trial.

Key characteristics of the sites and context should be reported.

The context of intervention delivery and any information that suggests contextual elements may impact the feasibility or future efficacy of the intervention should be clearly reported.

Where context is known or hypothesized to influence the implementation and/or feasibility of an intervention, including more than one site may be necessary.

Setting and contextual characteristics are known factors that can influence intervention outcomes. For PFS testing interventions that rely on a setting as part of the delivery process or are embedded naturally within existing settings, site selection and context become key factors to understand at the early stages of the design and evaluation of an intervention. Setting and context may represent static (e.g., hospital serving low-resource area) or dynamic (e.g., weather, day-to-day variability) characteristics [ 38 ]. Reasons why sites are selected in a PFS can include a range of pragmatic considerations. These include the need for representation of a diverse range of characteristics (e.g., geography, populations served), facilities/infrastructure required for the project (e.g., cell phone connectivity, low-resource settings), and proximity to the investigative team [ 39 , 40 , 41 , 42 , 43 , 44 , 45 ]. These decisions may also be based on the ability to refer sufficient numbers of participants at a given site [ 43 , 46 , 47 ]. Descriptions of the context and setting and how these might influence intervention outcomes should be clearly reported [ 38 , 48 , 49 ].

In some PFS, understanding setting complexity and how an intervention fits within a broader system may be the primary research questions that need to be answered prior to conducting a larger-scale trial. Studies investigating setting or context are useful for the identification of whether an intervention is appropriate or feasible to deliver for a given setting [ 50 , 51 , 52 , 53 ]. This allows for understanding uncertainties about the setting and how differences across settings may influence implementation [ 54 , 55 , 56 , 57 ]. In some situations, where an existing intervention is adapted to be delivered in a different setting, understanding how the intervention interacts with the new context becomes a key feasibility outcome to evaluate.

Stakeholder engagement and co-production

Stakeholder engagement and co-production refers to the use of partnerships with individuals, communities, and service providers to aid in the development and implementation of an intervention [ 58 ].

PFS should be, whenever possible, co-designed/co-created or informed by key stakeholder (e.g., community and professional) perspectives throughout all stages of design and implementation.

Whenever possible, pro-equity approaches that ensure the unique considerations and perspectives around an intervention’s acceptability, safety, etc., and participation in and ownership of research from minority and vulnerable populations, should be used.

The processes by which the PFS was co-designed, including who was consulted, why, when they were consulted, and how their input was obtained, should be clearly described.

Stakeholder engagement and co-production methods are commonly used in PFS to ensure the relevance of a number of intervention-related facets. These include the relevance of intervention materials, how an intervention is delivered, whether the content is appropriate, and if any important components are missing [ 59 , 60 , 61 ]. Employing stakeholder engagement and co-production methods can be useful to ensure ownership of the developed intervention by recipients and end-users [ 62 ]. Where these methods are employed, it is important to report who is involved in co-production (participants, interventionists, members of the public, other key stakeholders) and a rationale for their involvement in the process [ 63 , 64 , 65 ]. The process of engaging stakeholders in co-production can take many forms, including “think aloud”—commonly used for useability testing, questionnaires, and/or interviews [ 66 , 67 , 68 , 69 , 70 ]. What participants did during the co-production process, such as reviewing qualitative interviews or initial testing of intervention materials, should be reported. Details of how participants were engaged in the co-production (e.g., time dedicated, number of rounds of review/workshops, the total number of individuals involved) should also be included [ 71 , 72 ]. In some instances, it may be appropriate to describe details of the training required to facilitate a co-production process [ 61 ].

Theory usage

Theory usage refers to the utilization of any conceptual or theoretical model to inform aspects of the PFS that are mechanisms of change [ 8 ].

Researchers, where relevant, should include details about one or more behavior change theories (e.g., intervention activities, mechanisms) which informed aspects of the PFS, including whether components of the intervention are theoretically or practically informed.

The theoretical foundation of an intervention should be clearly stated. The components of an intervention may directly map on to one or more theories of change. These could be specific theories, mechanisms, or conceptual frameworks informed by practice. Theories of change should refer to intervention resources, activities, mechanisms, and intermediate and final outcomes. This information can be presented in the form of a logic model of change or conceptual frameworks depicting the theory of change or program theory [ 50 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 ]. Details of the theory of change and how this informed intervention development can be presented alongside pilot and/or feasibility outcomes, but could also be published separately, such as in a protocol overview [ 81 , 82 ].

Well-defined problem and aims

Well-defined problems and aims refers to the focused research questions/objectives used to guide the design, conduct, and analyses of PFS [ 8 ].

PFS should be guided by clear and focused research questions related primarily to the feasibility of the intervention and prospects of subsequent scale-up to a larger-scale trial. These well-formulated research questions should be answered by an appropriate and transparent methodology that uses both quantitative and qualitative data.

Where appropriate, the PFS proposal and report should define a clinically important public health problem for which researchers are designing, refining, or adapting an intervention.

PFS are designed primarily to answer key aspects regarding the feasibility of an intervention. These include addressing uncertainties about the intervention and the implications of the findings for larger-scale trials [ 83 ]. Questions of uncertainty are the basis for well-defined problems and aims of PFS. These can include understanding researchers’ access to the population of interest (recruitment); acceptability of randomization (for certain study designs); developing, refining, and finalizing intervention protocols; acceptability of the intervention for the target population; intervention deliverers and other key personnel; and other feasibility-related outcomes including fidelity, cost, equity, and cultural appropriateness [ 70 , 84 , 85 , 86 , 87 ].

In certain situations, the aims of a PFS can be more exploratory in nature. But this does not preclude the study from having a set of well-defined problems and aims. Examples may include learning about the assets, values, and/or history of the community in which an intervention could potentially be delivered and learning about the processes in which co-design and collaboration with community members could naturally take place prior to delivering an intervention.

Study design

Iteration and intervention refinement.

Iteration and intervention refinement refers to the re-testing of an intervention in PFS to further refine intervention components before scaling to a larger trial [ 88 ].

If the conclusion of the PFS is to make significant adjustments to either the study design or the intervention, then it should be acknowledged that the results do not justify proceeding further and a second PFS is necessary to establish feasibility before testing the intervention in a larger-scale, well-powered trial. Any potential changes (adaptations/tailoring) should be clearly documented along with information about how and why the changes are to be made (see 1.1. Adaptations and Tailoring ).

The decision to conduct multiple iterations of a PFS can be pragmatic or theoretical and based on factors including the perceived confidence the redesign will sufficiently address the identified problems.

Conclusions from a PFS should include whether the intervention, in its current form, is ready for a future trial or if modifications are needed (and if so, details of them), and whether they are substantial enough to warrant another PFS.

Iterations refer to the re-testing of an intervention in another PFS [ 89 , 90 , 91 , 92 ]. This can be done based upon findings from a previous PFS trial where minor and/or major adjustments to the intervention, its delivery, or other aspects of the study were found. Initial evaluations of an intervention may even pre-plan for multiple iterations. The iterations create a sequence of trialing and modifying prior to any well-powered trials. At the conclusion of a PFS, investigators can make the decision, based upon progression criteria and other findings, whether additional testing of the intervention needs to ensue prior to scale-up. This decision should be left to the interventionists and co-developers and be based on the evidence collected from the PFS, available resources, and time. Decisions can be pragmatic but also important are theoretical considerations that can inform whether or why alterations to the intervention may or may not result in anticipated or unanticipated changes.

Progression criteria

Progression criteria are a set of a priori benchmarks or thresholds regarding key feasibility markers that inform decisions about whether to proceed, to proceed with changes, or not to proceed from the PFS to a future study, either a main trial or another PFS [ 15 ].

PFS should include a set of progression criteria which are used to inform decisions about whether to proceed, proceed with changes, or not to proceed to a larger-scale study.

Progression criteria should be determined a priori and be based on either evidence from previously published/conducted research or a sound rationale provided.

Decisions on whether to proceed should also be informed by contextual, temporal, and partnership factors that evolve over the course of the pilot and/or feasibility.

Progression criteria should be made for feasibility metrics such as recruitment rate, retention/drop-out rate, acceptability, implementation/fidelity, and other appropriate feasibility indicators where appropriate.

Progression decisions can also include evidence of potential impact (see 5.2. Preliminary Impact ).

Progression criteria decisions can be in the form of a “Go/No Go” system or a “Stop Light” (red/amber/green) system, indicating no progression, progression with changes, or progression with no changes.

Deviations from the application of progression criteria may be justified if researchers are confident that a proposed solution will address the problem at a larger scale and can provide strong theoretical and/or empirical evidence to support their assertion (see 1.1. Adaptations/Tailoring ).

Across all feasibility metrics, some form of progression criteria thresholds and classification systems should be pre-defined [ 74 , 80 , 93 , 94 , 95 , 96 , 97 , 98 ]. The thresholds are commonly study- and intervention-specific, and these thresholds can be designated by investigators and any co-designers. Common classification schemes include red/amber/green and go/no-go. Often, these criteria are pre-registered and/or appear in protocol documents. Progression criteria can be used to gauge whether certain aspects of the intervention and its delivery along with other aspects of the study need to be modified. This information can be used to inform decisions about whether a subsequent test of the intervention should be conducted in another PFS (see 2.1. Iteration and Intervention Refinement ).

Randomization and control groups

Randomization refers to the process of using random chance to allocate units (individuals or settings/clusters) to one or more intervention conditions. Randomization can be used to separate units into distinct groups or randomization within a unit for when and what intervention(s) they may receive (order and timing). A control/comparator condition serves as the counterfactual. A control/comparator group is a group of participants (and/or settings/clusters) allocated to receive differing amounts, orders, or types of intervention(s) being tested [ 99 , 100 , 101 ]. A baseline period can serve as a control/comparator condition for studies employing single-arm or individual-level interventions (e.g., N-of-1) [ 102 ].

Not every PFS needs to include two or more groups or employ random allocation.

The presence of a control/comparator group or randomization can be included if it reflects the aims and objectives of the study.

Control groups can take numerous forms and should be reflective of the objectives of the study, the context within which the intervention is tested, and acceptability by the target population.

When randomization is employed, methods of randomization should be clearly described to ensure reproducibility.

If a control/comparator group is present, feasibility indicators collected on the intervention group should also be collected on the control group where appropriate (e.g., feasibility of data collection, acceptability of randomization, retention).

PFS can employ a range of designs. These include N-of-1 [ 103 ], micro-randomized trials [ 104 ], single-group [ 105 ], quasi-experimental [ 106 ], and multi-group/multi-setting designs [ 107 ]. Despite these design options, not every PFS needs to employ randomization or include more than one group. The use of randomization and multi-group design features should be based on the objectives of the PFS. Randomization in PFS can take the form of allocating groups to different interventions or varying levels of the same intervention (doses). Randomization can also take the form of within-person or group allocation of the timing and/or varying interventions participants may receive. Where multiple groups are included, “what” they receive (i.e., allocated to) should be based on the nature of the intervention and be consistent with conventions within the field of study. This can range from a purely no-treatment comparator to standard practice to alternate active interventions. Where some form of a comparator group is used, researchers should evaluate feasibility metrics to understand such things as the ability to retain those not receiving the intervention and acceptability of randomization. Incorporating either randomization or multiple groups can increase the scientific rigor of the PFS but is not necessary to evaluate most feasibility metrics of an intervention.

Scale-up refers to the process of delivering and evaluating an intervention in progressively larger studies, beginning with testing an intervention within one or more PFS and moving towards larger studies of the same, or similar, interventions. It is a “deliberate effort to increase the impact of successfully tested health intervention so as to benefit more people and foster policy and program development on a lasting basis” [ 108 , 109 ].

PFS should be designed with the intent for future testing of an intervention in large-scale trials and beyond.

Researchers should consider plans for later-phase research on the intervention and explain how information gathered from the PFS will be used to answer key questions surrounding the uncertainty of the intervention or the design or conduct of a progressively larger future study.

Issues regarding the adoption, implementation, and maintenance of the intervention over progressively larger studies can be considered at both the design and conduct phases of the PFS.

Efforts should be made to ensure key features of the PFS be similar to those in the future large-scale trial. These include the amount of support to implement the intervention, characteristics of who delivers the intervention, the target population, the duration under which the intervention is tested, and the measures employed.

Where differences are anticipated between pilot and/or feasibility testing and the larger-scale trial, a description of these differences should be provided along with a clear justification of how the changes may or may not impact the intervention.

PFS should be designed and conducted with the idea the information collected will be used to inform the testing of an intervention in progressively larger sample sizes and/or settings [ 85 , 110 , 111 , 112 , 113 , 114 , 115 , 116 ]. This implies researchers who conduct PFS intend to continue to refine and optimize an intervention for maximal impact along a translational science continuum [ 117 , 118 , 119 ]. With this in mind, understanding early on how an intervention could be delivered to progressively larger numbers of individuals and/or settings should be incorporated into the early stages of the design and conduct of PFS. Considerations for scaling can include characteristics of those who deliver an intervention, the resources required to train and deliver an intervention, and to whom an intervention is delivered. How these aspects can change as one progresses from commonly smaller-sized PFS to evaluating an intervention for broader population-level impact should inform what transpires in a PFS. Researchers should, therefore, consider whether what they can accomplish on a smaller scale can similarly be accomplished on a larger scale [ 120 , 121 ].

Conduct of trial

Measurement and data collection

Measurement and data collection refer to any tools, devices, instruments, personnel, and time required to assess feasibility or outcomes related to an intervention.

PFS can assess the feasibility and appropriateness of measurement and data collection procedures including the following:

How or if the data can be collected

The acceptability of the measurements and data collection procedures (e.g., burden)

If the measures are valid for the population/outcomes in question

Where applicable, measurements and data collection procedures should closely resemble those anticipated for the well-powered trial.

The reporting of measurement and data collection procedures should be sufficiently detailed to permit standardized data collection, including information about why the measurements were selected and how they were administered, scored, and interpreted.

Information about the feasibility and appropriateness of measurement and data collection procedures can consist of both quantitative and qualitative data sources.

The process of collecting outcome data in a PFS serves to demonstrate the feasibility of data collection methods—whether explicitly stated or not [ 122 ]. However, some PFS may be designed to answer whether outcome measures proposed for the larger-scale trial can be collected. This can include the ability to collect data using more invasive/burdensome methods (e.g., urine/hair samples, blood draws) [ 123 , 124 ]. Additional metrics associated with the feasibility of measurement and data collection may include determining rates of missing data, participant response rates, and any time/resource costs associated with data collection [ 125 , 126 , 127 ]. This information can be used to reduce participant burden and costs associated with data collection as well as refine protocols in the larger-scale trial [ 128 , 129 , 130 , 131 , 132 , 133 , 134 , 135 , 136 ].

Recruitment

Recruitment refers to the procedures used to identify and select potential participants (individuals and/or settings/clusters) and enroll them into a PFS. The recruitment rate is the proportion of eligible participants or settings/clusters who are enrolled at the baseline of an intervention trial compared to the invited/eligible target population [ 137 ].

Recruitment procedures should be clearly described, with any strategies designed to maximize recruitment fully detailed.

Information should include details of procedures used to recruit at the individual and setting/cluster levels, where appropriate.

Recruitment information should include the following, where appropriate:

Proportion of eligible units (e.g., individuals, settings) recruited

The start and end dates of the recruitment periods

Number of participants recruited per setting/cluster, overall, and number of settings/clusters

Number of potential participants screened, eligible, consented, and enrolled in the study

Reasons for non-recruitment/non-consent

Acceptability of recruitment strategies

Details should be provided about the recruitment strategies used, measures of their success, what worked, and what may need to be altered for future studies.

Participant recruitment is a key marker of intervention feasibility. Identifying optimal recruitment strategies in a PFS plays a critical role in determining whether the specified sample size can be achieved in the well-powered trial. Recruitment strategies may include opt-out methods (passive consent), telephone reminders, open designs (participants know what arm of the trial they are in), referrals, modalities of communication with potential participants (e.g., phone calls, emailing, text, mailings), convenient study location, and partnering with community members/settings [ 138 , 139 , 140 , 141 ]. The specific recruitment strategies used can influence the demographic makeup of participants. Different recruitment strategies can also yield varying amounts of eligible participants. In addition, each recruitment strategy has an associated cost. It may also be important to identify reasons why participants refused to participate in the study and record these reasons quantitatively and/or qualitatively. This information should be collected at the individual and/or setting level where appropriate. These can be important to establish during a PFS to optimize recruitment procedures in the larger-scale trial, especially in situations where there are uncertainties around recruiting the target population. At times, it may even be appropriate to formally test recruitment strategies, particularly when there is uncertainty about the best approach. For example, by embedding a “Study Within A Trial” (SWAT), researchers may gain answers to uncertainties around methodological decisions regarding a number of feasibility outcomes, including recruitment [ 142 , 143 ].

Retention (attrition/drop-out) is the proportion of enrolled participants who are present throughout the full length of the intervention [ 137 ].

Researchers conducting PFS should ensure retention rates are measured.

Where possible, assessments can be made to identify differences in retention across groups or intervention conditions.

Reasons why individuals leave a study can be collected and analyzed to investigate whether particular factors are associated with retention.

Procedures should clearly describe strategies used to assist with retaining participants during the delivery of the intervention and any post-intervention follow-up time periods, where appropriate.

Retention-related information can include both quantitative and qualitative data sources.

Retention is a commonly assessed marker of intervention feasibility. Retaining participants throughout an intervention is important to ensure participants receive the full dose of intervention components as designed and whether selective attrition is present. Retention-related information also helps to understand issues regarding missing data and low statistical power in future studies. Ultimately, retention is a marker of intervention viability. In other words, if participants do not want to receive an intervention it is unlikely to be impactful.

For a given intervention, a clear definition of retention should be reported. This can include participants staying for the duration of study-related procedures/measures (e.g., data collection), completing intervention components, and/or attendance at intervention sessions [ 22 , 92 , 128 , 144 ]. Depending on the nature of the intervention and the outcomes targeted, PFS may be designed specifically to address issues regarding retention in samples that have been historically challenging to engage/retain in interventions [ 145 , 146 ].

Retention strategies, such as flexible scheduling, reminders, compensation, consistency in study staff (continuity of relationships), gathering multiple contacts, thank you and birthday cards, and follow-up phone calls within a given period, can reduce the rate of participant drop-out [ 139 , 147 , 148 , 149 ]. Where dropouts occur, reasons for withdrawal from the study can be collected [ 128 , 150 ]. Factors influencing retention, both positively and negatively, including participant motivation/aspirations, expectations, the perceived need for an intervention, and accessibility of intervention (location delivered), can be collected from both participants and intervention deliverers [ 151 , 152 , 153 , 154 , 155 ].

Implementation of intervention

Acceptability.

Acceptability is a perception/notion that an intervention or various aspects of an intervention are favorable, agreeable, palatable, enjoyable, satisfactory, valued, appropriate from the perspectives of participants or communities receiving the intervention, and/or have a wider fit within a system. It relates to how users “feel” about an intervention [ 156 ].

Researchers should clearly define what is meant by “acceptability” for a given study, at what levels (e.g., individual, deliverer, setting) it will be assessed, and by what methods (e.g., surveys, interviews). This should be based on the nature of the intervention and its constituent components, target population, setting level characteristics, and key stakeholders.

Measures of acceptability can be pre-defined and included in both the PFS and large-scale trial stages.

Acceptability should be captured, at minimum, from the end user (intervention participants). Acceptability can also be captured from those involved with delivering the intervention, along with anyone else involved in the implementation process.

Acceptability, as defined for a given study, can be assessed for participants in control conditions where appropriate (e.g., acceptability of randomization to active comparator, acceptability of data collection procedures).

Researchers can use both quantitative (e.g., surveys) and qualitative (e.g., interviews) methods to assess acceptability.

In most behavioral interventions, it is important to understand whether those receiving an intervention, those delivering an intervention, and any other key individual(s) find the intervention, either in its entirely or in relevant parts, to be “acceptable” to inform whether the intervention would be used or tolerated. Acceptability encompasses a range of aspects related to impressions of an intervention. These can be gathered anytime along the intervention development continuum. At the earliest stages of conceptualization, prior to packaging and preliminary testing of an intervention, assessments of acceptability (preferences) can include participants’ views of whether the proposed intervention could be appropriate for addressing a given outcome, whether they (the participants) would be willing to adhere to an intervention, the suitability of intervention materials, or whether they perceive the intervention to be useful. During intervention delivery, ongoing assessment of likeability, satisfaction, metrics of engagement with an intervention, and utility can be collected periodically [ 45 , 157 , 158 , 159 ]. Once an intervention is completed, post-assessment markers of acceptability can include perceptions of the length or overall burden of the intervention, what strategies/components of an intervention were liked best, referral of the intervention to others, or whether the intervention met their (the recipients, deliverers, others) preferences/expectations. Where an intervention is delivered by individuals outside the intervention-development team, assessing their perspectives on the acceptability of an intervention may be necessary.

Assessments of acceptability can include both qualitative and quantitative measures. User-centered design [ 160 ] and “think aloud” protocols [ 161 ] can be used in the early stages of intervention conceptualization/formulization. Exit interviews, upon intervention completion, from recipients, deliverers, and other key individuals involved in the intervention, are often employed to evaluate markers of acceptability. Quantitative measures typically include items developed specifically for a given study. Alternatively, existing scales assessing acceptability can be used or modified accordingly for a given application [ 162 , 163 , 164 ]. Acceptability can also cover other aspects of the evaluation process of an intervention. This includes such areas as whether completing the proposed measures is feasible, acceptability of being randomized, or whether recipients were satisfied with the location where an intervention was delivered.

Fidelity is the degree to which an intervention is delivered as intended and the quality of that delivery [ 165 , 166 ].

Researchers should clearly define what is meant by “fidelity” for a given study, at what levels (e.g., individual, deliverer, setting) it will be assessed, and by what methods (e.g., surveys, interviews).

Measures of fidelity should be pre-defined with all intervention components listed.

Fidelity can consist of information about how an intervention will be delivered, for whom, what the intervention consists of, and when and where (context) the intervention will be delivered.

If strategies are used to encourage fidelity (e.g., a manualized intervention, feedback to those delivering the intervention), these should be reported.

Factors influencing fidelity can be assessed and, where appropriate, linked to feasibility outcomes.

Fidelity is often a primary marker of implementation. Assessment of an intervention’s fidelity provides key information regarding whether an intervention, either the testing of individual components or in their entirety, can be delivered as intended. In PFS where initial evaluations of an intervention are conducted, fidelity plays an important role in identifying whether the intervention can be delivered as intended. Evaluation of fidelity implies a working understanding of the intervention and some pre-planned, a priori expected delivery [ 167 , 168 ]. Measuring fidelity can be useful where adaptations (or changes) to the materials may take place (either planned or unplanned). Systematically documenting deviations from the original intervention can yield important insights into whether adaptations were beneficial or detrimental to the outcomes [ 169 ].

Fidelity can include many aspects of an intervention. These include adherence to intervention materials (what was done), quality of delivery (how it was done), and the dose of what was received [ 166 , 170 ]. Assessing fidelity can take many forms. This includes the creation of study-specific fidelity checklists which capture the presence of key components that should be delivered during an intervention (e.g., key material to be delivered in session one or a multi-session intervention) and how they were delivered [ 134 , 171 ]. Response ranges vary from present/absent, yes/no, to Likert-scaled items. Fidelity checklists can be completed either in real-time or reviewed later through the use of recorded video or audio of completed sessions [ 172 , 173 , 174 ]. Checklists can be completed by either someone external to the delivery agent via structured observations/recordings or completed by the delivery agent (e.g., self-report, logbooks) immediately following the delivery [ 175 , 176 , 177 ].

Qualitative interviews of delivery agents can also be conducted to gauge views regarding aspects of an intervention such as the training received to deliver, confidence in delivering, and any perceived barriers to delivering an intervention as planned [ 172 ]. Factors affecting fidelity can be collected to understand what, if anything, may influence departures from delivering an intervention as designed [ 22 , 132 , 173 , 178 ]. Common ways to encourage fidelity are through the use of a manualized package of procedures, training materials, and ongoing review of sessions accompanied by feedback.

Cost and resources

Costs and resources refer to the investments and assets required to develop, implement, and sustain an intervention [ 12 , 179 ].

PFS can include assessments of the costs and required resources of conducting an intervention.

In PFS costs and resources mIn PFS costs and resources may include the following:ay include the following:

Monetary costs associated with training, supervision, and recruitment of both stakeholders and participants, incentivization, facilities, materials, and intervention component development and delivery.

Opportunity costs/time demands associated with completing the intervention by participants and delivering the intervention by providers.

Researchers can collect information to determine the feasibility of measuring the costs associated with the intervention, with this information used to inform a more well-defined cost analysis/economic evaluation in a larger-scale trial.

Researchers should keep in mind that some costs associated with the intervention will be fixed (one-time costs) and some will be recurring during the successful scale-up and sustainment of the intervention.

For some PFS, collecting the costs associated with delivering an intervention may be necessary to inform a larger-scale trial. In PFS, this is often referred to as conducting an economic evaluation, costing, or cost analysis [ 125 , 180 , 181 , 182 , 183 ]. Studies may collect cost data to “rehearse” cost-effectiveness evaluations (economic evaluations) or evaluate the feasibility of collecting cost-related data [ 169 , 184 ]. Where cost data are collected, micro-costing approaches that inventory all associated costs with an intervention are often conducted and used to generate a total cost per unit estimate, often expressed as a cost per participant. Costs can be fixed, variable, or projected future estimates, and they may vary according to the desired fidelity and rigor of the implementation of the interventions. Common resources inventoried for cost include the costs of consumables, staff time, services received, transportation, room hires, and refreshments. Costs can be separated into the costs associated with the initial design/development, set up of the intervention, training of staff to deliver, and the costs associated with intervention delivery. The inclusion of cost data is not study-design specific and spans a wide range of designs from N of 1 to cluster randomized studies [ 185 , 186 , 187 ].

Statistical analysis

Sample size.

Sample size refers to the number of participants (or groups/clusters) in a given study [ 188 ].

The sample size of a PFS should be based on the feasibility objectives of the study.

Sample sizes do not have to be based upon a formal sample size calculation (i.e., power).

Sample sizes should be pre-specified and justified.

Sample size estimates should consider the representativeness of the target population or subgroup, setting, and other relevant contextual aspects that may influence how and why an intervention works.

Sample characteristics should be clearly described and may refer to individuals and/or clusters (e.g., churches, workplaces, neighborhoods, schools).

Where relevant, studies should clearly report factors impacting the sample size (e.g., availability of funds, time constraints).

Investigators are encouraged to report the a priori power achieved by the sample size selected for a PFS.

It is widely recognized that most PFS are underpowered to detect clinically significant/public health important effects in outcomes. Selecting the appropriate sample size for a PFS, however, can vary across studies based on the objectives. In some instances, formal power calculations can be conducted/presented, but one should avoid the temptation of presenting a PFS as being well-powered by assuming implausibly large effects and/or event rates and using non-relevant outcomes. Sample size justification can be made based on other factors including, but not limited to, the availability of resources, the number of potential participants within a given setting, representativeness of the sample to the target population, complexities regarding the intervention, or the experiences of the investigators working with the population/setting [ 189 , 190 , 191 , 192 , 193 ]. Regardless of the approach taken, researchers need to ensure they have sufficient numbers (i.e., sample size) to make informed decisions based on the feasibility metrics and objectives of a PFS and acknowledge any limitations that the usually small sample size confers.

Preliminary impact

Preliminary impact is the ability of an intervention, during a PFS to produce a desired or intended result [ 194 ].

PFS need not be powered to detect statistically significant differences in outcomes, but one or more outcomes, as appropriate to the research, can be assessed.

When outcomes are collected, changes in outcome data (e.g., estimated effect sizes) can be used to aid in decisions regarding the conduct of a subsequent larger-scale trial (e.g., sample size needed).

In many cases, it may be necessary to demonstrate an intervention “moves” outcomes in the appropriate direction and is not causing harm. In this scenario, it is recommended statistical testing can be performed but to avoid the interpretation of p values as conclusive evidence of an intervention’s impact in a larger-scale trial.

Interpretations of performed statistical tests should not include a justification for (or against) proceeding to a subsequent large-scale intervention or for making claims about the likely success of the study. Interpretations should help guide, but not dominate, the decision to proceed to a large-scale intervention.

Investigators should avoid misusing language such as “statistically significant” to describe their interpretation of outcomes from a PFS.

Where pilot and/or feasibility estimates of impact on primary, secondary, or tertiary outcomes are reported these should be pre-specified, with point estimates and a measure of variability reported for all time points.

For studies presenting both feasibility and outcome data, outcome data should be relegated to a secondary or exploratory focus.

In some circumstances, it may be appropriate to evaluate, in a preliminary/exploratory fashion, the potential impact of an intervention on proximal or distill outcomes in a PFS. Where outcomes are assessed and reported, researchers need to understand the evidence is neither definitive nor necessarily very indicative of an intervention’s impact within a larger-scale trial. Nevertheless, the evaluation of outcomes within a PFS can provide useful, additional information to help inform decisions about whether the intervention is ready to be tested at a larger scale. When reporting outcomes, researchers should avoid using misleading language centered on the presence or lack of “statistical significance”. All reported outcome assessments should be secondary to feasibility metrics, which are the primary focus of most PFS. Further, it is suggested that journals should not require by default outcome assessments and/or formal hypothesis testing for manuscripts that report on PFS nor base publishing decisions on the outcomes of potential efficacy analyses if reported.

Pre-registration and protocol publishing

Pre-registration and protocol publishing refers to an a priori process of documenting planned intervention design and analyses [ 195 ].

Pre-registration and a protocol made publicly available (via peer-reviewed journal, pre-print server, or other forms of public dissemination) contributes to transparency and ensures that changes between what is planned, what is conducted, and what is ultimately reported are communicated and justified.

We acknowledge there is a certain degree of flexibility when it comes to PFS between what is proposed and what actually transpires in the execution of the study. Pre-registration of PFS needs to balance the developmental/exploratory nature of these types of studies with the need to document and adhere to general protocols that are the foundation of rigorous and transparent science. The goal of pre-registration is not to create an inflexible scope of work that cannot adapt to uncertainties encountered in the study, but to communicate changes to a protocol and to justify why those changes were made.

Pre-registration of study objectives can be appropriate and at times required based upon funding stipulations. While some PFS are not pre-registered, many can be found on existing trial registries. These include Clinical Trials [ 196 ] and other emerging pre-print servers and open-science repositories, such as Open Science Framework [ 197 , 198 ]. Protocol publishing is also becoming increasingly common for PFS. Pre-registration and protocol publishing may help to provide details about a PFS as well as ensure deviations, although necessary at times, are clearly documented.

Study labeling

Study labeling refers to naming/presenting a PFS with appropriate naming conventions for the study being conducted [ 2 , 3 ].

At a minimum, researchers should make sure studies are clearly labeled to indicate their preliminary nature and reflect the aims and objectives of the study in both the title and abstract with either “pilot”, “feasibility”, “proof-of-concept”, “formative”, or other relevant label(s).

PFS should be clearly labeled to identify and separate them within the intervention development and evaluation literature. One of the benefits of clearly labeling PFS is the ease of identification of these types of studies to understand the evolution of behavioral interventions. Because PFS are often smaller in scale, clear identification also helps to distinguish these types of studies from studies that are small in scale and lack an emphasis on intervention development, refinement, and scaling.

A number of different taxonomies have been proposed to label these types of studies. However, we recognize researchers can and do use terms referring to preliminary studies interchangeably or utilize a combination of them to describe a single study [ 79 , 136 , 167 , 199 , 200 , 201 , 202 , 203 , 204 , 205 , 206 , 207 , 208 , 209 , 210 ]. In the absence of a universal consensus of terms, it is recommended investigators clearly label their PFS with one or more widely used terms that identify the preliminary nature of the study. These terms could include “pilot”, “feasibility”, “proof-of-concept”, “preliminary”, “evidentiary”, “vanguard”, and/or “exploratory”. Thus, investigators should identify the most appropriate term(s) that describe the objective of their study. This should consider the stage and number of tests/evaluations of an intervention.

Framework and guideline usage

The utilization of published frameworks/guidelines to guide the development, implementation, and reporting of PFS.

Where possible, researchers should choose an appropriate framework to structure PFS and use it to guide the design, conduct, analysis, and reporting of said study.

Findings from PFS should be disseminated in a way that adheres to reporting guidelines to facilitate transparency and allow for replication.

There are many existing guidelines, checklists, frameworks, and recommendations that can be useful for the design, conduct, implementation, analysis, and reporting of PFS [ 9 , 211 ]. The use of these publications is associated with higher study quality and reporting [ 9 ]. Guidelines include those developed specifically for PFS and also include those designed outside of the preliminary study context but are highly relevant to many aspects of PFS. Investigators should be familiar with existing guidance and utilize it appropriately, based on the specific objectives of their PFS.

PFS play a pivotal role in the development, refinement, implementation, and sustainability of successful behavioral interventions. This is evidenced by their emphasis on funding agencies [ 4 , 212 , 213 , 214 , 215 , 216 ] and depiction within translational science frameworks [ 117 , 118 , 217 , 218 ]. We identified 161 publications offering some form of guidelines, checklists, frameworks, or recommendations for PFS. Through a Delphi study utilizing expert perspectives, we developed a comprehensive set of considerations which span the continuum of development, conduct, implementation, evaluation, and reporting of behavioral intervention PFS. We believe this will serve as a valuable resource for researchers in the behavioral sciences.

Continued challenges with PFS

While this consolidation of considerations for PFS was developed for broad applicability, there were strong opposing views among the Delphi study participants on some of the considerations that represent continued challenges with PFS. The most striking opposing opinions were observed within the “statistical analysis” theme and were present in both the “sample size” and “preliminary impact” considerations. For example, several respondents in the Delphi study believed sample size estimates for a larger-scale trial can be informed by the estimated intervention effect sizes generated from a PFS, and formal hypothesis testing can be performed and associated p values interpreted in a preliminary study. Other respondents expressed strong opinions that the sample of a PFS need not be representative of the target population. Conversely, the vast majority of respondents agreed that sample size justifications should be based on the feasibility objectives of a given PFS and argued against hypothesis testing (i.e., formal statistical testing and interpretation of p values) during the early phases of intervention development. There have been arguments made for reporting confidence intervals instead of p values for any non-feasibility-related outcomes assessed during PFS [ 219 , 220 , 221 , 222 ]. However, respondents of our Delphi study were quick to point out there is little practical difference between the use of p values or confidence intervals, especially if the PFS is underpowered from the start.

Opposing views were identified throughout the Delphi process for other considerations as well, including “study labeling” and “pre-registration and protocol publishing”. For study labeling, some respondents appreciated the distinction between “pilot”, “feasibility”, and other “preliminary study” terminology, while others worried that these distinctions were not well known and may cause undue confusion. Many participants of the Delphi study indicated they would rather there be no distinction, voicing concerns that adopting rigid taxonomies would create research silos and hinder cross-purpose innovation. Ultimately, we chose not to take a definitive stance on this issue, but rather make researchers aware they should be labeling PFS in some way to aid in the identification of these types of studies. On the topic of pre-registration and protocol publishing, some Delphi respondents argued that pre-registration and protocol publishing for PFS was asking too much and that this type of work should be reserved only for larger-scale trials. Others fully supported the idea of pre-registration and protocol publishing for PFS, arguing it aids in transparency and reproducibility. Again, these are decisions ultimately left up to the researchers conducting PFS, but it is likely that registration will be increasingly requested and enforced (e.g., by funders). The lack of registration of all PFS means that one cannot understand the totality of the efforts that are made in that space for developing and assessing the feasibility of an intervention.

It is important to understand that what may be viewed as common and accepted practice may not be widely held everywhere and the reasons for this vary according to country, funder, and disciplinary norms. It may be that differing opinions stem from differences between what commonly accepted/promoted translational science frameworks espouse and the realities of conducting PFS, which are often conducted with limited resources and abbreviated timelines. In addition, there may be different levels of expectations about what is proposed in these frameworks and the expectations of funding agencies and grant reviewers [ 223 ]. Such disagreements can prove problematic for behavioral scientists when seeking funding or wanting to publish findings from their PFS. Reconciliation on these topics is unlikely, and perhaps unnecessary, yet it is important to acknowledge what can and cannot be accomplished by a PFS. We believe appropriately tending to these issues throughout all phases of design, conduct, interpretation, and reporting should help preemptively dissuade critiques that could stymie the progress of intervention development and implementation.

Progress for PFS

While disagreements were noted for a few considerations, most respondents agreed on the content of most topics. For example, participants of the Delphi study agreed that feasibility outcomes, including recruitment, retention, acceptability, and fidelity should take priority over preliminary impact and should be used and presented as the primary outcomes of PFS. This also aligns with developing well-designed problems and aims of PFS, most of which should answer questions regarding uncertainties (feasibility) of an intervention. Respondents also agreed progression criteria are useful when developing and deploying PFS, although some recommended caution on the use of progression criteria that are too rigid when making decisions about scaling up PFS to the next stage. Finally, and perhaps most salient, participants agreed on the importance of PFS as a critical step in successful large-scale intervention development and implementation. However, one cannot exclude the presence of selection bias in favor of the importance of PFS among authors who have authored guidelines on them and even more so among authors who responded to our surveys.

Use of the considerations

We believe the considerations in this paper span the continuum of PFS, from development to reporting, and will be useful for researchers planning to conduct their very first PFS to well-seasoned interventionists. We envision these consolidated considerations being used in practice and as an educational tool for trainees. On a broader scale, we are hopeful this consolidation may improve PFS in the future, reducing research waste and leading to the development of high-quality, scalable behavioral interventions with maximal reach and public health impact. In addition to the considerations themselves, we provide a crosswalk of all published guidelines, checklists, frameworks, and recommendations related to PFS in Additional file 2 in an effort to amplify the voices of experts in this field. Researchers reading this study and those who want to know more about a particular consideration are encouraged to utilize the crosswalk located in Additional file 1 to identify further reading, which may provide more specific guidance on a particular topic. While not the focus of this consolidation, we also believe many of the considerations are cross-cutting with large-scale implementation and dissemination research. Researchers doing this type of work may look to certain considerations to guide aspects of their larger-scale study as well.

Strengths and limitations

These consolidated considerations have several strengths. First, they were created based on information gathered from 161 published guidelines, checklists, frameworks, and recommendations on the topic of PFS. We relied on authors from these very same 161 publications to voice their opinions about the most important PFS-related topics via a three-round Delphi study. The total sample of participants across three rounds of the Delphi process represented over 35% of the 161 publications. Participants had, on average, 23 years of experience since their terminal degree, representing a collective 1150 years of experience across respondents. Moreover, we supplement this consolidation with a review of those 161 guidelines, checklists, frameworks, and recommendations, creating one of the largest collective sources of information on PFS published to date. This study is not without limitations. While we had a moderate representation of Delphi participants across publications, we were only able to recruit 10% (50 out of 496 identified authors) of our target population for the Delphi process. Further, while there was an equal distribution of males and females, the sample was largely White. Other than age and years of terminal degree, we did not collect other demographic information on the Delphi participants, although the median year of publication for the publications represented in our sample was slightly more recent (2015) than the total sample of possible publications (2013) from which authors were sampled. For the considerations themselves, there is still no true consensus on many of the topics presented. Differences of opinion were observed throughout the Delphi process and can be found across the published literature. Despite this, we believe the consolidated considerations could be a valuable resource for behavioral interventionists conducting PFS on a broad range of public health topics.

This is one of the first studies to attempt to garner consensus on a broad range of considerations regarding PFS for the behavioral sciences. We leveraged expert opinion from researchers who have published PFS-related guidelines, checklists, frameworks, and recommendations on a wide range of topics and distilled this knowledge into a valuable and universal resource for researchers conducting PFS. We identified 20 considerations for PFS, which fall into six categories, including intervention design , study design , conduct of trial , implementation of intervention , statistical analysis , and reporting . We also provide a list of the available publications on each of the specific considerations for further reading and use and have aligned these publications with the considerations set forth in this paper. Researchers may use these considerations alongside the previously published literature to guide decision making about all aspects of PFS, with the hope of creating and disseminating interventions with broad public health impact.

Availability of data and materials

The datasets used and analyzed during the current study are freely available at https://osf.io/kyft7/ .

Indig D, Lee K, Grunseit A, Milat A, Bauman A. Pathways for scaling up public health interventions. BMC Public Health. 2018;18(1):68. https://doi.org/10.1186/s12889-017-4572-5 .

Article   Google Scholar  

Eldridge SM, Lancaster GA, Campbell MJ, et al. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. Plos One. 2016;11(3):e0150205. https://doi.org/10.1371/journal.pone.0150205 . Lazzeri C, ed.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Bond C, Lancaster GA, Campbell M, et al. Pilot and feasibility studies: extending the conceptual framework. Pilot Feasibil Stud. 2023;9(1):24. https://doi.org/10.1186/s40814-023-01233-1 .

National Institutes of Health. NIH Planning Grant Program (R34) .; 2019. https://grants.nih.gov/grants/funding/r34.htm . Accessed 13 July, 2023

Moher D, Altman DG, Schulz KF, Simera I. How to develop a reporting guideline. In: Moher D, Altman DG, Schulz KF, Simera I, Wager E, eds. Guidelines for Reporting Health Research: A User’s Manual. John Wiley & Sons, Ltd; 2014:14–21. https://doi.org/10.1002/9781118715598.ch2

Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. Plos Med. 2010;7(2):e1000217. https://doi.org/10.1371/journal.pmed.1000217 .

Article   PubMed   PubMed Central   Google Scholar  

Eldridge SM, Chan CL, Campbell MJ, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ. Published online October 24 2016;i5239. https://doi.org/10.1136/bmj.i5239

O’Cathain A, Croot L, Duncan E, et al. Guidance on how to develop complex interventions to improve health and healthcare. BMJ Open. 2019;9(8):e029954. https://doi.org/10.1136/bmjopen-2019-029954 .

Pfledderer CD, Von Klinggraeff L, Burkart S, et al. Use of guidelines, checklists, frameworks, and recommendations in behavioral intervention preliminary studies and associations with reporting comprehensiveness: a scoping bibliometric review. Pilot Feasibil Stud. 2023;9:161. https://doi.org/10.1186/s40814-023-01389-w .

Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. Published online September 29, 2008:a1655. https://doi.org/10.1136/bmj.a1655

Bowen DJ, Kreuter M, Spring B, et al. How we design feasibility studies. Am J Prev Med. 2009;36(5):452–7. https://doi.org/10.1016/j.amepre.2009.02.002 .

Pearson N, Naylor PJ, Ashe MC, Fernandez M, Yoong SL, Wolfenden L. Guidance for conducting feasibility and pilot studies for implementation trials. Pilot Feasibil Stud. 2020;6(1):167. https://doi.org/10.1186/s40814-020-00634-w .

Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique: Delphi survey technique. J Adv Nurs. 2000;32(4):1008–15. https://doi.org/10.1046/j.1365-2648.2000.t01-1-01567.x .

Article   CAS   PubMed   Google Scholar  

Keeney S, Hasson F, McKenna HP. The Delphi Technique in Nursing and Health Research. Oxford: Wiley-Blackwell; 2011.

Mellor K, Albury C, Dutton SJ, Eldridge S, Hopewell S. Recommendations for progression criteria during external randomised pilot trial design, conduct, analysis and reporting. Pilot Feasibil Stud. 2023;9(1):59. https://doi.org/10.1186/s40814-023-01291-5 . s40814-023-01291-01295.

Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci. 2013;8(1):65. https://doi.org/10.1186/1748-5908-8-65 .

Swindle T, Martinez A, Børsheim E, Andres A. Adaptation of an exercise intervention for pregnant women to community-based delivery: a study protocol. BMJ Open. 2020;10(9):e038582. https://doi.org/10.1136/bmjopen-2020-038582 .

Sizemore KM, Gray S, Wolfer C, et al. A proof of concept pilot examining feasibility and acceptability of the positively healthy just-in-time adaptive, ecological momentary, intervention among a sample of sexual minority men living with HIV. J Happiness Stud. 2022;23(8):4091–118. https://doi.org/10.1007/s10902-022-00587-2 .

Duarte N, Hughes SL, Paúl C. Cultural adaptation and specifics of the fit & strong! program in Portugal. Transl Behav Med. 2019;9(1):67–75. https://doi.org/10.1093/tbm/iby003 .

Article   PubMed   Google Scholar  

Eylem O, Van Straten A, De Wit L, Rathod S, Bhui K, Kerkhof AJFM. Reducing suicidal ideation among Turkish migrants in the Netherlands and in the UK: the feasibility of a randomised controlled trial of a guided online intervention. Pilot Feasibil Stud. 2021;7(1):30. https://doi.org/10.1186/s40814-021-00772-9 .

Mathews E, Thomas E, Absetz P, et al. Cultural adaptation of a peer-led lifestyle intervention program for diabetes prevention in India: the Kerala diabetes prevention program (K-DPP). BMC Public Health. 2017;17(1):974. https://doi.org/10.1186/s12889-017-4986-0 .

Zullig LL, McCant F, Silberberg M, Johnson F, Granger BB, Bosworth HB. Changing CHANGE: adaptations of an evidence-based telehealth cardiovascular disease risk reduction intervention. Transl Behav Med. 2018;8(2):225–32. https://doi.org/10.1093/tbm/ibx030 .

Hill JL, Zoellner JM, You W, et al. Participatory development and pilot testing of iChoose: an adaptation of an evidence-based paediatric weight management program for community implementation. BMC Public Health. 2019;19(1):122. https://doi.org/10.1186/s12889-019-6450-9 .

Sánchez-Franco S, Arias LF, Jaramillo J, et al. Cultural adaptation of two school-based smoking prevention programs in Bogotá Colombia. Transl Behav Med. 2021;11(8):1567–78. https://doi.org/10.1093/tbm/ibab019 .

Lloyd J, Bjornstad G, Borek A, et al. Healthy parent carers programme: mixed methods process evaluation and refinement of a health promotion intervention. BMJ Open. 2021;11(8):e045570. https://doi.org/10.1136/bmjopen-2020-045570 .

Meiksin R, Allen E, Crichton J, et al. Protocol for pilot cluster RCT of project respect: a school-based intervention to prevent dating and relationship violence and address health inequalities among young people. Pilot Feasibil Stud. 2019;5(1):13. https://doi.org/10.1186/s40814-019-0391-z .

Sebire SJ, Banfield K, Jago R, et al. A process evaluation of the PLAN-A intervention (Peer-Led physical Activity iNtervention for Adolescent girls). BMC Public Health. 2019;19(1):1203. https://doi.org/10.1186/s12889-019-7545-z .

Mueller S, Soriano D, Boscor A, et al. MANTRA: development and localization of a mobile educational health game targeting low literacy players in low and middle income countries. BMC Public Health. 2020;20(1):1171. https://doi.org/10.1186/s12889-020-09246-8 .

Gaume J, Grazioli VS, Paroz S, Fortini C, Bertholet N, Daeppen JB. Developing a brief motivational intervention for young adults admitted with alcohol intoxication in the emergency department – results from an iterative qualitative design. Plos One. 2021;16(2):e0246652. https://doi.org/10.1371/journal.pone.0246652 . Alam A (Neeloy, ed).

Siu AL, Zimbroff RM, Federman AD, et al. The effect of adapting Hospital at Home to facilitate implementation and sustainment on program drift or voltage drop. BMC Health Serv Res. 2019;19(1):264. https://doi.org/10.1186/s12913-019-4063-8 .

Martin S, Rassi C, Antonio V, et al. Evaluating the feasibility and acceptability of a community dialogue intervention in the prevention and control of schistosomiasis in Nampula province, Mozambique. Plos One. 2021;16(8):e0255647. https://doi.org/10.1371/journal.pone.0255647 . Diemert DJ, ed.

Ellis-Hill C, Thomas S, Gracey F, et al. HeART of Stroke: randomised controlled, parallel-arm, feasibility study of a community-based arts and health intervention plus usual care compared with usual care to increase psychological well-being in people following a stroke. BMJ Open. 2019;9(3):e021098. https://doi.org/10.1136/bmjopen-2017-021098 .

Walton A, Nahum-Shani I, Crosby L, Klasnja P, Murphy S. Optimizing digital integrated care via micro-randomized trials. Clin Pharmacol Ther. 2018;104(1):53–8. https://doi.org/10.1002/cpt.1079 .

Odukoya OO, Manortey S, Takemoto M, Alder S, Okuyemi KS. Body, Soul and Spirit, an adaptation of two evidence-based interventions to promote physical activity and healthy eating among adults in churches in Lagos Nigeria: a three-arm cluster randomized controlled pilot trial. Pilot Feasibil Stud. 2020;6(1):59. https://doi.org/10.1186/s40814-020-00600-6 .

Malden S, Hughes AR, Gibson AM, et al. Adapting the ToyBox obesity prevention intervention for use in Scottish preschools: protocol for a feasibility cluster randomised controlled trial. BMJ Open. 2018;8(10):e023707. https://doi.org/10.1136/bmjopen-2018-023707 .

Istanboulian L, Rose L, Yunusova Y, Dale CM. Protocol for a mixed method acceptability evaluation of a codesigned bundled COmmunication intervention for use in the adult ICU during the COVID-19 PandEmic: the COPE study. BMJ Open. 2021;11(9):e050347. https://doi.org/10.1136/bmjopen-2021-050347 .

McKay H, Nettlefold L, Bauman A, et al. Implementation of a co-designed physical activity program for older adults: positive impact when delivered at scale. BMC Public Health. 2018;18(1):1289. https://doi.org/10.1186/s12889-018-6210-2 .

Klasnja P, Smith S, Seewald NJ, et al. Efficacy of contextually tailored suggestions for physical activity: a micro-randomized optimization trial of HeartSteps. Ann Behav Med. 2019;53(6):573–82. https://doi.org/10.1093/abm/kay067 .

Murray E, Daff K, Lavida A, Henley W, Irwin J, Valabhji J. Evaluation of the digital diabetes prevention programme pilot: uncontrolled mixed-methods study protocol. BMJ Open. 2019;9(5):e025903. https://doi.org/10.1136/bmjopen-2018-025903 .

Rodrigues AmorimAdegboye A, Cocate PG, Benaim C, et al. Recruitment of low-income pregnant women into a dietary and dental care intervention: lessons from a feasibility trial. Trials. 2020;21(1):244. https://doi.org/10.1186/s13063-020-4142-5 .

Northridge ME, Metcalf SS, Yi S, Zhang Q, Gu X, Trinh-Shevrin C. A protocol for a feasibility and acceptability study of a participatory, multi-level, dynamic intervention in urban outreach centers to improve the oral health of low-income Chinese Americans. Front Public Health. 2018;6:29. https://doi.org/10.3389/fpubh.2018.00029 .

Nicolson GH, Hayes C, Darker C. A theory-based multicomponent intervention to reduce occupational sedentary behaviour in professional male workers: protocol for a cluster randomised crossover pilot feasibility study. Pilot Feasibil Stud. 2020;6(1):175. https://doi.org/10.1186/s40814-020-00716-9 .

Thomas DSK, Bull S, Nyanza EC, Hampanda K, Liedtke M, Ngallaba SE. An mHealth pilot designed to increase the reach of prevention of mother-to-child transmission of HIV (PMTCT) across the treatment cascade in a resource-constrained setting in Tanzania. Plos One. 2019;14(2):e0212305. https://doi.org/10.1371/journal.pone.0212305 . Biemba G, ed.

Lemanska A, Poole K, Griffin BA, et al. Community pharmacy lifestyle intervention to increase physical activity and improve cardiovascular health of men with prostate cancer: a phase II feasibility study. BMJ Open. 2019;9(6):e025114. https://doi.org/10.1136/bmjopen-2018-025114 .

Logie CH, Daniel C, Newman PA, Loutfy MR. An HIV/STI prevention intervention for internally displaced women in Leogane, Haiti: study protocol for an N-of-1 pilot study. BMJ Open. 2012;2(4):e001634. https://doi.org/10.1136/bmjopen-2012-001634 .

Coulman E, Hastings R, Gore N, et al. The Early Positive Approaches to Support (E-PAtS) study: study protocol for a feasibility cluster randomised controlled trial of a group programme (E-PAtS) for family caregivers of young children with intellectual disability. Pilot Feasibil Stud. 2020;6(1):147. https://doi.org/10.1186/s40814-020-00689-9 .

Njuguna IN, Beima-Sofie K, Mburu CW, et al. Adolescent transition to adult care for HIV-infected adolescents in Kenya (ATTACH): study protocol for a hybrid effectiveness-implementation cluster randomised trial. BMJ Open. 2020;10(12):e039972. https://doi.org/10.1136/bmjopen-2020-039972 .

Bradford N, Condon P, Pitt E, Tyack Z, Alexander K. Optimising symptom management in children with cancer using a novel mobile phone application: protocol for a controlled hybrid effectiveness implementation trial (RESPONSE). BMC Health Serv Res. 2021;21(1):942. https://doi.org/10.1186/s12913-021-06943-x .

Maughan-Brown B, Smith P, Kuo C, et al. A conditional economic incentive fails to improve linkage to care and antiretroviral therapy initiation among HIV-positive adults in Cape Town South Africa. AIDS Patient Care STDs. 2018;32(2):70–8. https://doi.org/10.1089/apc.2017.0238 .

Kramer F, Labudek S, Jansen CP, et al. Development of a conceptual framework for a group-based format of the Lifestyle-integrated Functional Exercise (gLiFE) programme and its initial feasibility testing. Pilot Feasibil Stud. 2020;6(1):6. https://doi.org/10.1186/s40814-019-0539-x .

Doody P, Lord JM, Whittaker AC. Assessing the feasibility and impact of an adapted resistance training intervention, aimed at improving the multi-dimensional health and functional capacity of frail older adults in residential care settings: protocol for a feasibility study. Pilot Feasibil Stud. 2019;5(1):86. https://doi.org/10.1186/s40814-019-0470-1 .

Morris AS, Murphy RC, Shepherd SO, Healy GN, Edwardson CL, Graves LEF. A multi-component intervention to sit less and move more in a contact centre setting: a feasibility study. BMC Public Health. 2019;19(1):292. https://doi.org/10.1186/s12889-019-6615-6 .

Millar A, Tip L, Lennon R, et al. The introduction of mindfulness groups to a psychiatric rehabilitation in-patient setting: a feasibility study. BMC Psychiatry. 2020;20(1):322. https://doi.org/10.1186/s12888-020-02725-7 .

Rasheed MA, Bharuchi V, Mughis W, Hussain A. Development and feasibility testing of a play-based psychosocial intervention for reduced patient stress in a pediatric care setting: experiences from Pakistan. Pilot Feasibil Stud. 2021;7(1):63. https://doi.org/10.1186/s40814-021-00781-8 .

Azar KMJ, Nasrallah C, Szwerinski NK, et al. Implementation of a group-based diabetes prevention program within a healthcare delivery system. BMC Health Serv Res. 2019;19(1):694. https://doi.org/10.1186/s12913-019-4569-0 .

Tsakos G, Brocklehurst PR, Watson S, et al. Improving the oral health of older people in care homes (TOPIC): a protocol for a feasibility study. Pilot Feasibil Stud. 2021;7(1):138. https://doi.org/10.1186/s40814-021-00872-6 .

Clemson L, Laver K, Jeon YH, et al. Implementation of an evidence-based intervention to improve the wellbeing of people with dementia and their carers: study protocol for ‘Care of People with dementia in their Environments (COPE)’ in the Australian context. BMC Geriatr. 2018;18(1):108. https://doi.org/10.1186/s12877-018-0790-7 .

Hawkins J, Madden K, Fletcher A, et al. Development of a framework for the co-production and prototyping of public health interventions. BMC Public Health. 2017;17(1):689. https://doi.org/10.1186/s12889-017-4695-8 .

Gillespie J, Hughes A, Gibson AM, Haines J, Taveras E, Reilly JJ. Protocol for Healthy Habits Happy Homes (4H) Scotland: feasibility of a participatory approach to adaptation and implementation of a study aimed at early prevention of obesity. BMJ Open. 2019;9(6):e028038. https://doi.org/10.1136/bmjopen-2018-028038 .

Griva K, Chia JMX, Goh ZZS, et al. Effectiveness of a brief positive skills intervention to improve psychological adjustment in patients with end-stage kidney disease newly initiated on haemodialysis: protocol for a randomised controlled trial (HED-Start). BMJ Open. 2021;11(9):e053588. https://doi.org/10.1136/bmjopen-2021-053588 .

Chudleigh J, Holder P, Moody L, et al. Process evaluation of co-designed interventions to improve communication of positive newborn bloodspot screening results. BMJ Open. 2021;11(8):e050773. https://doi.org/10.1136/bmjopen-2021-050773 .

Maindal HT, Timm A, Dahl-Petersen IK, et al. Systematically developing a family-based health promotion intervention for women with prior gestational diabetes based on evidence, theory and co-production: the Face-it study. BMC Public Health. 2021;21(1):1616. https://doi.org/10.1186/s12889-021-11655-2 .

Bray EA, George A, Everett B, Salamonson Y, Ramjan L. Protocol for developing a healthcare transition intervention for young people with spinal cord injuries using a participatory action research approach. BMJ Open. 2021;11(7):e053212. https://doi.org/10.1136/bmjopen-2021-053212 .

Goffe L, Hillier-Brown F, Hildred N, et al. Feasibility of working with a wholesale supplier to co-design and test acceptability of an intervention to promote smaller portions: an uncontrolled before-and-after study in British Fish & Chip shops. BMJ Open. 2019;9(2):e023441. https://doi.org/10.1136/bmjopen-2018-023441 .

Livings R, Naylor JM, Gibson K, et al. Implementation of a community-based, physiotherapy-led, multidisciplinary model of care for the management of knee osteoarthritis: protocol for a feasibility study. BMJ Open. 2020;10(7):e039152. https://doi.org/10.1136/bmjopen-2020-039152 .

Guagliano JM, Brown HE, Coombes E, et al. The development and feasibility of a randomised family-based physical activity promotion intervention: the Families Reporting Every Step to Health (FRESH) study. Pilot Feasibil Stud. 2019;5(1):21. https://doi.org/10.1186/s40814-019-0408-7 .

Kassavou A, Houghton V, Edwards S, Brimicombe J, Sutton S. Development and piloting of a highly tailored digital intervention to support adherence to antihypertensive medications as an adjunct to primary care consultations. BMJ Open. 2019;9(1):e024121. https://doi.org/10.1136/bmjopen-2018-024121 .

Payne Riches S, Piernas C, Aveyard P, Sheppard JP, Rayner M, Jebb SA. The Salt Swap intervention to reduce salt intake in people with high blood pressure: protocol for a feasibility randomised controlled trial. Trials. 2019;20(1):584. https://doi.org/10.1186/s13063-019-3691-y .

Degroote L, Van Dyck D, De Bourdeaudhuij I, De Paepe A, Crombez G. Acceptability and feasibility of the mHealth intervention ‘MyDayPlan’ to increase physical activity in a general adult population. BMC Public Health. 2020;20(1):1032. https://doi.org/10.1186/s12889-020-09148-9 .

Bodschwinna D, Lorenz I, Bauereiss N, Gündel H, Baumeister H, Hoenig K. PartnerCARE—a psycho-oncological online intervention for partners of patients with cancer: study protocol for a randomised controlled feasibility trial. BMJ Open. 2020;10(10):e035599. https://doi.org/10.1136/bmjopen-2019-035599 .

Lowthian JA, Green M, Meyer C, et al. Being Your Best: protocol for a feasibility study of a codesigned approach to reduce symptoms of frailty in people aged 65 years or more after transition from hospital. BMJ Open. 2021;11(3):e043223. https://doi.org/10.1136/bmjopen-2020-043223 .

Goff LM, Moore AP, Rivas C, Harding S. Healthy Eating and Active Lifestyles for Diabetes (HEAL-D): study protocol for the design and feasibility trial, with process evaluation, of a culturally tailored diabetes self-management programme for African-Caribbean communities. BMJ Open. 2019;9(2):e023733. https://doi.org/10.1136/bmjopen-2018-023733 .

Vandervelde S, Scheepmans K, Milisen K, et al. Reducing the use of physical restraints in home care: development and feasibility testing of a multicomponent program to support the implementation of a guideline. BMC Geriatr. 2021;21(1):77. https://doi.org/10.1186/s12877-020-01946-5 .

Simpson S, Wyke S, Mercer SW. Adaptation of a mindfulness-based intervention for incarcerated young men: a feasibility study. Mindfulness. 2019;10(8):1568–78. https://doi.org/10.1007/s12671-018-1076-z .

Jumbe S, James WY, Madurasinghe V, et al. Evaluating NHS Stop Smoking Service engagement in community pharmacies using simulated smokers: fidelity assessment of a theory-based intervention. BMJ Open. 2019;9(5):e026841. https://doi.org/10.1136/bmjopen-2018-026841 .

Langford R, Jago R, White J, et al. A physical activity, nutrition and oral health intervention in nursery settings: process evaluation of the NAP SACC UK feasibility cluster RCT. BMC Public Health. 2019;19(1):865. https://doi.org/10.1186/s12889-019-7102-9 .

Appel JM, Fullerton K, Hennessy E, et al. Design and methods of shape up under 5: integration of systems science and community-engaged research to prevent early childhood obesity. Plos One. 2019;14(8):e0220169. https://doi.org/10.1371/journal.pone.0220169 . Nkomazana O, ed.

Myers B, Carney T, Browne FA, Wechsberg WM. A trauma-informed substance use and sexual risk reduction intervention for young South African women: a mixed-methods feasibility study. BMJ Open. 2019;9(2):e024776. https://doi.org/10.1136/bmjopen-2018-024776 .

Brewer LC, Hayes SN, Caron AR, et al. Promoting cardiovascular health and wellness among African-Americans: community participatory approach to design an innovative mobile-health intervention. Plos One. 2019;14(8):e0218724. https://doi.org/10.1371/journal.pone.0218724 . Newton RL, ed.

Mitchell KR, Purcell C, Simpson SA, et al. Feasibility study of peer-led and school-based social network Intervention (STASH) to promote adolescent sexual health. Pilot Feasibil Stud. 2021;7(1):125. https://doi.org/10.1186/s40814-021-00835-x .

on behalf of the REACH-HF investigators, Greaves CJ, Wingham J, et al. Optimising self-care support for people with heart failure and their caregivers: development of the Rehabilitation Enablement in Chronic Heart Failure (REACH-HF) intervention using intervention mapping. Pilot Feasibil Stud. 2016;2(1):37. https://doi.org/10.1186/s40814-016-0075-x .

Lang CC, Smith K, Wingham J, et al. A randomised controlled trial of a facilitated home-based rehabilitation intervention in patients with heart failure with preserved ejection fraction and their caregivers: the REACH-HFpEF Pilot Study. BMJ Open. 2018;8(4):e019649. https://doi.org/10.1136/bmjopen-2017-019649 .

Abuhaloob L, Helles N, Mossey P, Freeman R. An ADePT evaluation for incorporating the TIPPS periodontal health intervention into primary care antenatal programmes to enhance infant birth weight in Palestine: a feasibility study. Pilot Feasibil Stud. 2021;7(1):91. https://doi.org/10.1186/s40814-021-00827-x .

Davis M, Wolk CB, Jager-Hyman S, et al. Implementing nudges for suicide prevention in real-world environments: project INSPIRE study protocol. Pilot Feasibil Stud. 2020;6(1):143. https://doi.org/10.1186/s40814-020-00686-y .

Barnett M, Miranda J, Kia-Keating M, Saldana L, Landsverk J, Lau AS. Developing and evaluating a lay health worker delivered implementation intervention to decrease engagement disparities in behavioural parent training: a mixed methods study protocol. BMJ Open. 2019;9(7):e028988. https://doi.org/10.1136/bmjopen-2019-028988 .

Lucas-Thompson R, Seiter N, Broderick PC, et al. Moving 2 Mindful (M2M) study protocol: testing a mindfulness group plus ecological momentary intervention to decrease stress and anxiety in adolescents from high-conflict homes with a mixed-method longitudinal design. BMJ Open. 2019;9(11):e030948. https://doi.org/10.1136/bmjopen-2019-030948 .

Clouse K, Phillips TK, Camlin C, et al. CareConekta: study protocol for a randomized controlled trial of a mobile health intervention to improve engagement in postpartum HIV care in South Africa. Trials. 2020;21(1):258. https://doi.org/10.1186/s13063-020-4190-x .

McCrabb S, Mooney K, Elton B, Grady A, Yoong SL, Wolfenden L. How to optimise public health interventions: a scoping review of guidance from optimisation process frameworks. BMC Public Health. 2020;20(1):1849. https://doi.org/10.1186/s12889-020-09950-5 .

Neuhaus M, Healy GN, Fjeldsoe BS, et al. Iterative development of Stand Up Australia: a multi-component intervention to reduce workplace sitting. Int J Behav Nutr Phys Act. 2014;11(1):21. https://doi.org/10.1186/1479-5868-11-21 .

Fanning J, Brooks AK, Ip E, et al. A mobile health behavior intervention to reduce pain and improve health in older adults with obesity and chronic pain: the MORPH pilot trial. Front Digit Health. 2020;2:598456. https://doi.org/10.3389/fdgth.2020.598456 .

Larsen B, Greenstadt ED, Olesen BL, Marcus BH, Godino J, Zive MM. An mHealth physical activity intervention for latina adolescents: iterative design of the Chicas Fuertes study. JMIR Form Res. 2021;5(6):e26195. https://doi.org/10.2196/26195 .

Pagoto S, Tulu B, Agu E, Waring ME, Oleski JL, Jake-Schoffman DE. Using the Habit App for weight loss problem solving: development and feasibility study. JMIR MHealth UHealth. 2018;6(6):e145. https://doi.org/10.2196/mhealth.9801 .

Forsyth R, Purcell C, Barry S, et al. Peer-led intervention to prevent and reduce STI transmission and improve sexual health in secondary schools (STASH): protocol for a feasibility study. Pilot Feasibil Stud. 2018;4(1):180. https://doi.org/10.1186/s40814-018-0354-9 .

Nixon AC, Bampouras TM, Gooch HJ, et al. The EX-FRAIL CKD trial: a study protocol for a pilot randomised controlled trial of a home-based EXercise programme for pre-frail and FRAIL, older adults with chronic kidney disease. BMJ Open. 2020;10(6):e035344. https://doi.org/10.1136/bmjopen-2019-035344 .

Woodford J, Wikman A, Cernvall M, et al. Study protocol for a feasibility study of an internet-administered, guided, CBT-based, self-help intervention (ENGAGE) for parents of children previously treated for cancer. BMJ Open. 2018;8(6):e023708. https://doi.org/10.1136/bmjopen-2018-023708 .

Huang H, Yang P, Xue J, et al. Evaluating the individualized treatment of traditional Chinese medicine: a pilot study of N-of-1 trials. Evid Based Complement Alternat Med. 2014;2014:1–10. https://doi.org/10.1155/2014/148730 .

Hernandez LM, Wetter DW, Kumar S, Sutton SK, Vinci C. Smoking cessation using wearable sensors: protocol for a microrandomized trial. JMIR Res Protoc. 2021;10(2):e22877. https://doi.org/10.2196/22877 .

Militello L, Sobolev M, Okeke F, Adler DA, Nahum-Shani I. Digital prompts to increase engagement with the Headspace App and for stress regulation among parents: feasibility study. JMIR Form Res. 2022;6(3):e30606. https://doi.org/10.2196/30606 .

Kang M, Ragan BG, Park JH. Issues in outcomes research: an overview of randomization techniques for clinical trials. J Athl Train. 2008;43(2):215–21. https://doi.org/10.4085/1062-6050-43.2.215 .

Nair B. Clinical trial designs. Indian Dermatol Online J. 2019;10(2):193–201. https://doi.org/10.4103/idoj.IDOJ_475_18 .

Kendall JM. Designing a research project: randomised controlled trials and their principles. Emerg Med J. 2003;20(2):164–8. https://doi.org/10.1136/emj.20.2.164 .

D’Agostino RB, Kwan H. Measuring effectiveness What to expect without a randomized control group. Med Care. 1995;33(4 Suppl):AS95-105.

PubMed   Google Scholar  

Kravitz RL, Aguilera A, Chen EJ, et al. Feasibility, acceptability, and influence of mHealth-supported N-of-1 trials for enhanced cognitive and emotional well-being in US volunteers. Front Public Health. 2020;8:260. https://doi.org/10.3389/fpubh.2020.00260 .

Golbus JR, Dempsey W, Jackson EA, Nallamothu BK, Klasnja P. Microrandomized trial design for evaluating just-in-time adaptive interventions through mobile health technologies for cardiovascular disease. Circ Cardiovasc Qual Outcomes. 2021;14(2):e006760. https://doi.org/10.1161/CIRCOUTCOMES.120.006760 .

Pratt D, Mitchell H, Fitzpatrick L, Lea J. A single-group pilot feasibility and acceptability study of the broad minded affective coping technique for suicidal adults in crisis. J Behav Cogn Ther. 2022;32(4):290–6. https://doi.org/10.1016/j.jbct.2022.07.002 .

Dierick F, Bouché AF, Guérin S, et al. Quasi-experimental pilot study to improve mobility and balance in recurrently falling nursing home residents by voluntary non-targeted side-stepping exercise intervention. BMC Geriatr. 2022;22(1):1006. https://doi.org/10.1186/s12877-022-03696-y .

Agarwal G, Gaber J, Richardson J, et al. Pilot randomized controlled trial of a complex intervention for diabetes self-management supported by volunteers, technology, and interprofessional primary health care teams. Pilot Feasibil Stud. 2019;5(1):118. https://doi.org/10.1186/s40814-019-0504-8 .

Milat AJ, King L, Bauman AE, Redman S. The concept of scalability: increasing the scale and potential adoption of health promotion interventions into policy and practice. Health Promot Int. 2013;28(3):285–98. https://doi.org/10.1093/heapro/dar097 .

Smith JM, De Graft-Johnson J, Zyaee P, Ricca J, Fullerton J. Scaling up high-impact interventions: how is it done? Int J Gynecol Obstet. 2015;130:S4–10. https://doi.org/10.1016/j.ijgo.2015.03.010 .

Wulandari LPL, Kaldor J, Guy R. Uptake and acceptability of assisted and unassisted HIV self-testing among men who purchase sex in brothels in Indonesia: a pilot intervention study. BMC Public Health. 2020;20(1):730. https://doi.org/10.1186/s12889-020-08812-4 .

Thomas ED, Zohura F, Hasan MT, et al. Formative research to scale up a handwashing with soap and water treatment intervention for household members of diarrhea patients in health facilities in Dhaka, Bangladesh (CHoBI7 program). BMC Public Health. 2020;20(1):831. https://doi.org/10.1186/s12889-020-08727-0 .

Gagnon MP, Ndiaye MA, Larouche A, et al. Optimising patient active role with a user-centred eHealth platform (CONCERTO+) in chronic diseases management: a study protocol for a pilot cluster randomised controlled trial. BMJ Open. 2019;9(4):e028554. https://doi.org/10.1136/bmjopen-2018-028554 .

Bowden JL, Egerton T, Hinman RS, et al. Protocol for the process and feasibility evaluations of a new model of primary care service delivery for managing pain and function in patients with knee osteoarthritis (PARTNER) using a mixed methods approach. BMJ Open. 2020;10(2):e034526. https://doi.org/10.1136/bmjopen-2019-034526 .

Whittaker SL, Taylor NF, Hill KD, Ekegren CL, Brusco NK. Self-managed occupational therapy and physiotherapy for adults receiving inpatient rehabilitation (‘My Therapy’): protocol for a mixed-methods process evaluation. BMC Health Serv Res. 2021;21(1):810. https://doi.org/10.1186/s12913-021-06463-8 .

Bloom I, Welch L, Vassilev I, et al. Findings from an exploration of a social network intervention to promote diet quality and health behaviours in older adults with COPD: a feasibility study. Pilot Feasibil Stud. 2020;6(1):15. https://doi.org/10.1186/s40814-020-0553-z .

Jackson C, Huque R, Ahmed F, et al. Children Learning About Second-hand Smoke (CLASS II): a mixed methods process evaluation of a school-based intervention. Pilot Feasibil Stud. 2021;7(1):112. https://doi.org/10.1186/s40814-021-00853-9 .

Czajkowski SM, Powell LH, Adler N, et al. From ideas to efficacy: The ORBIT model for developing behavioral treatments for chronic diseases. Health Psychol. 2015;34(10):971–82. https://doi.org/10.1037/hea0000161 .

Onken LS, Carroll KM, Shoham V, Cuthbert BN, Riddle M. Reenvisioning clinical science: unifying the discipline to improve the public health. Clin Psychol Sci. 2014;2(1):22–34. https://doi.org/10.1177/2167702613497932 .

Onken L. Implementation science at the national institute on aging: the principles of it. Public Policy Aging Rep. 2022;32(1):39–41. https://doi.org/10.1093/ppar/prab034 . Kaskie B, ed.

Beets MW, Von Klinggraeff L, Weaver RG, Armstrong B, Burkart S. Small studies, big decisions: the role of pilot/feasibility studies in incremental science and premature scale-up of behavioral interventions. Pilot Feasibil Stud. 2021;7(1):173. https://doi.org/10.1186/s40814-021-00909-w .

Von Klinggraeff L, Dugger R, Okely AD, et al. Early-stage studies to larger-scale trials: investigators’ perspectives on scaling-up childhood obesity interventions. Pilot Feasibil Stud. 2022;8(1):31. https://doi.org/10.1186/s40814-022-00991-8 .

Pressman A, Law H, Stahl R, et al. Conducting a pilot randomized controlled trial of community-based mindfulness-based stress reduction versus usual care for moderate-to-severe migraine: protocol for the Mindfulness and Migraine Study (M&M). Trials. 2019;20(1):257. https://doi.org/10.1186/s13063-019-3355-y .

on behalf of the PD_Manager consortium, Antonini A, Gentile G, et al. Acceptability to patients, carers and clinicians of an mHealth platform for the management of Parkinson’s disease (PD_Manager): study protocol for a pilot randomised controlled trial. Trials. 2018;19(1):492. https://doi.org/10.1186/s13063-018-2767-4 .

Cardwell K, Smith SM, Clyne B, et al. Evaluation of the General Practice Pharmacist (GPP) intervention to optimise prescribing in Irish primary care: a non-randomised pilot study. BMJ Open. 2020;10(6):e035087. https://doi.org/10.1136/bmjopen-2019-035087 .

Adams N, Skelton DA, Howel D, et al. Feasibility of trial procedures for a randomised controlled trial of a community based group exercise intervention for falls prevention for visually impaired older people: the VIOLET study. BMC Geriatr. 2018;18(1):307. https://doi.org/10.1186/s12877-018-0998-6 .

Sahota P, Christian M, Day R, Cocks K. The feasibility and acceptability of a primary school-based programme targeting diet and physical activity: the PhunkyFoods Programme. Pilot Feasibil Stud. 2019;5(1):152. https://doi.org/10.1186/s40814-019-0542-2 .

Huberty J, Matthews J, Leiferman J, Cacciatore J, Gold KJ. A study protocol of a three-group randomized feasibility trial of an online yoga intervention for mothers after stillbirth (The Mindful Health Study). Pilot Feasibil Stud. 2018;4(1):12. https://doi.org/10.1186/s40814-017-0162-7 .

Yang MJ, Sutton SK, Hernandez LM, et al. A Just-In-Time Adaptive intervention (JITAI) for smoking cessation: feasibility and acceptability findings. Addict Behav. 2023;136:107467. https://doi.org/10.1016/j.addbeh.2022.107467 .

Karkar R, Schroeder J, Epstein DA, et al. TummyTrials: a feasibility study of using self-experimentation to detect individualized food triggers. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM; 2017:6850–6863. https://doi.org/10.1145/3025453.3025480

Munson SA, Schroeder J, Karkar R, Kientz JA, Chung CF, Fogarty J. The importance of starting with goals in N-of-1 studies. Front Digit Health. 2020;2:3. https://doi.org/10.3389/fdgth.2020.00003 .

McGrattan AM, McEvoy CT, Vijayakumar A, et al. A mixed methods pilot randomised controlled trial to develop and evaluate the feasibility of a Mediterranean diet and lifestyle education intervention ‘THINK-MED’ among people with cognitive impairment. Pilot Feasibil Stud. 2021;7(1):3. https://doi.org/10.1186/s40814-020-00738-3 .

Davies M, Kristunas CA, Huddlestone L, et al. Increasing uptake of structured self-management education programmes for type 2 diabetes in a primary care setting: a feasibility study. Pilot Feasibil Stud. 2020;6(1):71. https://doi.org/10.1186/s40814-020-00606-0 .

Jobst S, Leppla L, Köberich S. A self-management support intervention for patients with atrial fibrillation: a randomized controlled pilot trial. Pilot Feasibil Stud. 2020;6(1):87. https://doi.org/10.1186/s40814-020-00624-y .

Tarrant M, Carter M, Dean SG, et al. Singing for people with aphasia (SPA): results of a pilot feasibility randomised controlled trial of a group singing intervention investigating acceptability and feasibility. BMJ Open. 2021;11(1):e040544. https://doi.org/10.1136/bmjopen-2020-040544 .

Mama SK, Bhuiyan N, Bopp MJ, McNeill LH, Lengerich EJ, Smyth JM. A faith-based mind–body intervention to improve psychosocial well-being among rural adults. Transl Behav Med. 2020;10(3):546–54. https://doi.org/10.1093/tbm/ibz136 .

Stephenson A, Garcia-Constantino M, Murphy MH, McDonough SM, Nugent CD, Mair JL. The “Worktivity” mHealth intervention to reduce sedentary behaviour in the workplace: a feasibility cluster randomised controlled pilot study. BMC Public Health. 2021;21(1):1416. https://doi.org/10.1186/s12889-021-11473-6 .

Hallingberg B, Turley R, Segrott J, et al. Exploratory studies to decide whether and how to proceed with full-scale evaluations of public health interventions: a systematic review of guidance. Pilot Feasibil Stud. 2018;4(1):104. https://doi.org/10.1186/s40814-018-0290-8 .

Chatters R, Newbould L, Sprange K, et al. Recruitment of older adults to three preventative lifestyle improvement studies. Trials. 2018;19(1):121. https://doi.org/10.1186/s13063-018-2482-1 .

DeFrank G, Singh S, Mateo KF, et al. Key recruitment and retention strategies for a pilot web-based intervention to decrease obesity risk among minority youth. Pilot Feasibil Stud. 2019;5(1):109. https://doi.org/10.1186/s40814-019-0492-8 .

Greidanus MA, De Rijk AE, De Boer AGEM, et al. A randomised feasibility trial of an employer-based intervention for enhancing successful return to work of cancer survivors (MiLES intervention). BMC Public Health. 2021;21(1):1433. https://doi.org/10.1186/s12889-021-11357-9 .

Kroska EB, Hoel S, Victory A, et al. Optimizing an acceptance and commitment therapy microintervention via a mobile app with two cohorts: protocol for micro-randomized trials. JMIR Res Protoc. 2020;9(9):e17086. https://doi.org/10.2196/17086 .

Treweek S, Bevan S, Bower P, et al. Trial forge guidance 1: what is a Study Within A Trial (SWAT)? Trials. 2018;19(1):139. https://doi.org/10.1186/s13063-018-2535-5 .

Treweek S, Bevan S, Bower P, et al. Trial forge guidance 2: how to decide if a further Study Within A Trial (SWAT) is needed. Trials. 2020;21(1):33. https://doi.org/10.1186/s13063-019-3980-5 .

Vetrovsky T, Cupka J, Dudek M, et al. A pedometer-based walking intervention with and without email counseling in general practice: a pilot randomized controlled trial. BMC Public Health. 2018;18(1):635. https://doi.org/10.1186/s12889-018-5520-8 .

Smith JD, Berkel C, Rudo-Stern J, et al. The family check-up 4 Health (FCU4Health): applying implementation science frameworks to the process of adapting an evidence-based parenting program for prevention of pediatric obesity and excess weight gain in primary care. Front Public Health. 2018;6:293. https://doi.org/10.3389/fpubh.2018.00293 .

Dillingham R, Ingersoll K, Flickinger TE, et al. PositiveLinks: a mobile health intervention for retention in HIV care and clinical outcomes with 12-month follow-up. AIDS Patient Care STDs. 2018;32(6):241–50. https://doi.org/10.1089/apc.2017.0303 .

Orkin A, Campbell D, Handford C, et al. Protocol for a mixed-methods feasibility study for the surviving opioid overdose with naloxone education and resuscitation (SOONER) randomised control trial. BMJ Open. 2019;9(11):e029436. https://doi.org/10.1136/bmjopen-2019-029436 .

Krutsinger DC, Yadav KN, Cooney E, Brooks S, Halpern SD, Courtright KR. A pilot randomized trial of five financial incentive strategies to increase study enrollment and retention rates. Contemp Clin Trials Commun. 2019;15:100390. https://doi.org/10.1016/j.conctc.2019.100390 .

Phillips S, Kanter J, Mueller M, et al. Feasibility of an mHealth self-management intervention for children and adolescents with sickle cell disease and their families. Transl Behav Med. 2021;11(3):724–32. https://doi.org/10.1093/tbm/ibaa132 .

Nikles J, Mitchell GK, Schluter P, et al. Aggregating single patient (n-of-1) trials in populations where recruitment and retention was difficult: the case of palliative care. J Clin Epidemiol. 2011;64(5):471–80. https://doi.org/10.1016/j.jclinepi.2010.05.009 .

Taylor G, Aveyard P, Bartlem K, et al. IntEgrating Smoking Cessation treatment As part of usual Psychological care for dEpression and anxiety (ESCAPE): protocol for a randomised and controlled, multicentre, acceptability, feasibility and implementation trial. Pilot Feasibil Stud. 2019;5(1):16. https://doi.org/10.1186/s40814-018-0385-2 .

Clouse K, Mongwenyana C, Musina M, et al. Acceptability and feasibility of a financial incentive intervention to improve retention in HIV care among pregnant women in Johannesburg South Africa. AIDS Care. 2018;30(4):453–60. https://doi.org/10.1080/09540121.2017.1394436 .

Ortiz JA, Smith BW, Shelley BM, Erickson KS. Adapting mindfulness to engage latinos and improve mental health in primary care: a pilot study. Mindfulness. 2019;10(12):2522–31. https://doi.org/10.1007/s12671-019-01229-0 .

Fuchs JD, Stojanovski K, Vittinghoff E, et al. A mobile health strategy to support adherence to antiretroviral preexposure prophylaxis. AIDS Patient Care STDs. 2018;32(3):104–11. https://doi.org/10.1089/apc.2017.0255 .

Myers ND, Lee S, Bateman AG, et al. Accelerometer-based assessment of physical activity within the fun for wellness online behavioral intervention: protocol for a feasibility study. Pilot Feasibil Stud. 2019;5(1):73. https://doi.org/10.1186/s40814-019-0455-0 .

Gooding K, Phiri M, Peterson I, Parker M, Desmond N. Six dimensions of research trial acceptability: how much, what, when, in what circumstances, to whom and why? Soc Sci Med. 2018;213:190–8. https://doi.org/10.1016/j.socscimed.2018.07.040 .

Hoel S, Victory A, Sagorac Gruichich T, et al. A mixed-methods analysis of mobile ACT responses from two cohorts. Front Digit Health. 2022;4:869143. https://doi.org/10.3389/fdgth.2022.869143 .

Coughlin LN, Nahum-Shani I, Bonar EE, et al. Toward a just-in-time adaptive intervention to reduce emerging adult alcohol use: testing approaches for identifying when to intervene. Subst Use Misuse. 2021;56(14):2115–25. https://doi.org/10.1080/10826084.2021.1972314 .

Mande A, Moore SL, Banaei-Kashani F, Echalier B, Bull S, Rosenberg MA. Assessment of a mobile health iPhone app for semiautomated self-management of chronic recurrent medical conditions using an N-of-1 trial framework: feasibility pilot study. JMIR Form Res. 2022;6(4):e34827. https://doi.org/10.2196/34827 .

Dopp AR, Parisi KE, Munson SA, Lyon AR. A glossary of user-centered design strategies for implementation experts. Transl Behav Med. 2019;9(6):1057–64. https://doi.org/10.1093/tbm/iby119 .

Rose H, McKinley J, eds. The Routledge Handbook of Research Methods in Applied Linguistics. London: Routledge; 2020.

Lewis JR. The system usability scale: past, present, and future. Int J Human-Computer Interact. 2018;34(7):577–90. https://doi.org/10.1080/10447318.2018.1455307 .

Larsen DL, Attkisson CC, Hargreaves WA, Nguyen TD. Assessment of client/patient satisfaction: development of a general scale. Eval Program Plann. 1979;2(3):197–207. https://doi.org/10.1016/0149-7189(79)90094-6 .

Sidani S, Epstein DR, Bootzin RR, Moritz P, Miranda J. Assessment of preferences for treatment: validation of a measure: PREFERENCE MEASURE. Res Nurs Health. 2009;32(4):419–31. https://doi.org/10.1002/nur.20329 .

An M, Dusing SC, Harbourne RT, Sheridan SM, START-Play Consortium. What really works in intervention? Using fidelity measures to support optimal outcomes. Phys Ther. 2020;100(5):757–65. https://doi.org/10.1093/ptj/pzaa006 .

Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3–4):327–50. https://doi.org/10.1007/s10464-008-9165-0 .

Rivard M, Mello C, Mestari Z, et al. Using prevent teach reinforce for young children to manage challenging behaviors in public specialized early intervention services for autism. J Autism Dev Disord. 2021;51(11):3970–88. https://doi.org/10.1007/s10803-020-04856-y .

Stagg HR, Abubakar I, Campbell CN, et al. IMPACT study on intervening with a manualised package to achieve treatment adherence in people with tuberculosis: protocol paper for a mixed-methods study, including a pilot randomised controlled trial. BMJ Open. 2019;9(12):e032760. https://doi.org/10.1136/bmjopen-2019-032760 .

Kilbride C, Scott DJM, Butcher T, et al. Rehabilitation via HOMe Based gaming exercise for the Upper-limb post Stroke (RHOMBUS): protocol of an intervention feasibility trial. BMJ Open. 2018;8(11):e026620. https://doi.org/10.1136/bmjopen-2018-026620 .

Dumas JE, Lynch AM, Laughlin JE, Phillips Smith E, Prinz RJ. Promoting intervention fidelity. Am J Prev Med. 2001;20(1):38–47. https://doi.org/10.1016/S0749-3797(00)00272-5 .

Walton H, Spector A, Roberts A, et al. Developing strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia: a mixed methods study. BMC Med Res Methodol. 2020;20(1):153. https://doi.org/10.1186/s12874-020-01006-x .

Nomikos PA, Hall M, Fuller A, et al. Fidelity assessment of nurse-led non-pharmacological package of care for knee pain in the package development phase of a feasibility randomised controlled trial based in secondary care: a mixed methods study. BMJ Open. 2021;11(7):e045242. https://doi.org/10.1136/bmjopen-2020-045242 .

Hall M, Fuller A, Nomikos PA, et al. East Midlands knee pain multiple randomised controlled trial cohort study: cohort establishment and feasibility study protocol. BMJ Open. 2020;10(9):e037760. https://doi.org/10.1136/bmjopen-2020-037760 .

Horne JC, Hooban KE, Lincoln NB, Logan PA. Regaining Confidence after Stroke (RCAS): a feasibility randomised controlled trial (RCT). Pilot Feasibil Stud. 2019;5(1):96. https://doi.org/10.1186/s40814-019-0480-z .

Porter KJ, Brock DJ, Estabrooks PA, et al. SIPsmartER delivered through rural, local health districts: adoption and implementation outcomes. BMC Public Health. 2019;19(1):1273. https://doi.org/10.1186/s12889-019-7567-6 .

Watson A, Timperio A, Brown H, Hesketh KD. Process evaluation of a classroom active break (ACTI-BREAK) program for improving academic-related and physical activity outcomes for students in years 3 and 4. BMC Public Health. 2019;19(1):633. https://doi.org/10.1186/s12889-019-6982-z .

Hind D, Drabble SJ, Arden MA, et al. Feasibility study for supporting medication adherence for adults with cystic fibrosis: mixed-methods process evaluation. BMJ Open. 2020;10(10):e039089. https://doi.org/10.1136/bmjopen-2020-039089 .

Tibbitts B, Porter A, Sebire SJ, et al. Action 3:30R: process evaluation of a cluster randomised feasibility study of a revised teaching assistant-led extracurricular physical activity intervention for 8 to 10 year olds. BMC Public Health. 2019;19(1):1111. https://doi.org/10.1186/s12889-019-7347-3 .

Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38(2):65–76. https://doi.org/10.1007/s10488-010-0319-7 .

Wijana MB, Feldman I, Ssegonja R, Enebrink P, Ghaderi A. A pilot study of the impact of an integrated individual- and family therapy model for self-harming adolescents on overall healthcare consumption. BMC Psychiatry. 2021;21(1):374. https://doi.org/10.1186/s12888-021-03375-z .

Roberts JL, Williams J, Griffith GM, et al. Soles of the feet meditation intervention for people with intellectual disability and problems with anger and aggression—a feasibility study. Mindfulness. 2020;11(10):2371–85. https://doi.org/10.1007/s12671-020-01454-y .

Fallin-Bennett A, Lofwall M, Waters T, et al. Behavioral and Enhanced Perinatal Intervention (B-EPIC): a randomized trial targeting tobacco use among opioid dependent pregnant women. Contemp Clin Trials Commun. 2020;20:100657. https://doi.org/10.1016/j.conctc.2020.100657 .

Krebs P, Sherman SE, Wilson H, et al. Text2Connect: a health system approach to engage tobacco users in quitline cessation services via text messaging. Transl Behav Med. 2020;10(1):292–301. https://doi.org/10.1093/tbm/ibz033 .

Bailey DP, Edwardson CL, Pappas Y, et al. A randomised-controlled feasibility study of the REgulate your SItting Time (RESIT) intervention for reducing sitting time in individuals with type 2 diabetes: study protocol. Pilot Feasibil Stud. 2021;7(1):76. https://doi.org/10.1186/s40814-021-00816-0 .

Karnon J, Qizilbash N. Economic evaluation alongsiden-of-1 trials: getting closer to the margin. Health Econ. 2001;10(1):79–82. https://doi.org/10.1002/1099-1050(200101)10:1%3c79::AID-HEC567%3e3.0.CO;2-Z .

Dichter MN, Berg A, Hylla J, et al. Evaluation of a multi-component, non-pharmacological intervention to prevent and reduce sleep disturbances in people with dementia living in nursing homes (MoNoPol-sleep): study protocol for a cluster-randomized exploratory trial. BMC Geriatr. 2021;21(1):40. https://doi.org/10.1186/s12877-020-01997-8 .

Williams J, Fairbairn E, McGrath R, et al. A feasibility hybrid II randomised controlled trial of volunteer ‘Health Champions’ supporting people with serious mental illness manage their physical health: study protocol. Pilot Feasibil Stud. 2021;7(1):116. https://doi.org/10.1186/s40814-021-00854-8 .

Teare MD, Dimairo M, Shephard N, Hayman A, Whitehead A, Walters SJ. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials. 2014;15(1):264. https://doi.org/10.1186/1745-6215-15-264 .

Azariah S, Saxton P, Franklin R, Forster R, Werder S, Jenkins R. NZPrEP Demonstration Project: protocol for an open-label, single-arm trial of HIV pre-exposure prophylaxis (PrEP) to determine feasibility, acceptability, adverse and behavioural effects of PrEP provision to gay and bisexual men in publicly funded sexual health clinics in Auckland, New Zealand. BMJ Open. 2019;9(6):e026363. https://doi.org/10.1136/bmjopen-2018-026363 .

Dawkins L, Bauld L, Ford A, et al. A cluster feasibility trial to explore the uptake and use of e-cigarettes versus usual care offered to smokers attending homeless centres in Great Britain. Plos One. 2020;15(10):e0240968. https://doi.org/10.1371/journal.pone.0240968 . Leroyer C, ed.

Farragher JF, Thomas C, Ravani P, Manns B, Elliott MJ, Hemmelgarn BR. Protocol for a pilot randomised controlled trial of an educational programme for adults on chronic haemodialysis with fatigue (Fatigue-HD). BMJ Open. 2019;9(7):e030333. https://doi.org/10.1136/bmjopen-2019-030333 .

Shani P, Raeesi K, Walter E, et al. Qigong mind-body program for caregivers of cancer patients: design of a pilot three-arm randomized clinical trial. Pilot Feasibil Stud. 2021;7(1):73. https://doi.org/10.1186/s40814-021-00793-4 .

Croke A, Moriarty F, Boland F, et al. Integrating clinical pharmacists within general practice: protocol for a pilot cluster randomised controlled trial. BMJ Open. 2021;11(3):e041541. https://doi.org/10.1136/bmjopen-2020-041541 .

Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011;45(5):626–9. https://doi.org/10.1016/j.jpsychires.2010.10.008 .

American Psychological Association. Preregistration .; 2021. https://www.apa.org/pubs/journals/resources/preregistration . Accessed 12 July, 2023

National Library of Medicine. Clinical trials. Published online 2023. https://clinicaltrials.gov/ . Accessed 12 July, 2023

Center for Open Science. Open Science Framework. Published online 2023. https://osf.io/ . Accessed 12 July, 2023

Foster ED, Deardorff A. Open Science Framework (OSF). J Med Libr Assoc. 2017;105(2):203. https://doi.org/10.5195/jmla.2017.88 .

Article   PubMed Central   Google Scholar  

Murphy ME, McSharry J, Byrne M, et al. Supporting care for suboptimally controlled type 2 diabetes mellitus in general practice with a clinical decision support system: a mixed methods pilot cluster randomised trial. BMJ Open. 2020;10(2):e032594. https://doi.org/10.1136/bmjopen-2019-032594 .

Torres S, Sales CMD, Guerra MP, Simões MP, Pinto M, Vieira FM. Emotion-focused cognitive behavioral therapy in comorbid obesity with binge eating disorder: a pilot study of feasibility and long-term outcomes. Front Psychol. 2020;11:343. https://doi.org/10.3389/fpsyg.2020.00343 .

Ritchwood TD, Massa C, Kamanga G, Pettifor A, Hoffman I, Corneli A. Understanding of perceived infectiousness and its influence on sexual behavior among individuals with acute HIV infection in Lilongwe, Malawi (HPTN 062). AIDS Educ Prev. 2020;32(3):260–70. https://doi.org/10.1521/aeap.2020.32.3.260 .

Eaton AD, Chan Carusone S, Craig SL, et al. The ART of conversation: feasibility and acceptability of a pilot peer intervention to help transition complex HIV-positive people from hospital to community. BMJ Open. 2019;9(3):e026674. https://doi.org/10.1136/bmjopen-2018-026674 .

Holliday R, Preshaw PM, Ryan V, et al. A feasibility study with embedded pilot randomised controlled trial and process evaluation of electronic cigarettes for smoking cessation in patients with periodontitis. Pilot Feasibil Stud. 2019;5(1):74. https://doi.org/10.1186/s40814-019-0451-4 .

Wilchesky M, Mueller G, Morin M, et al. The OptimaMed intervention to reduce inappropriate medications in nursing home residents with severe dementia: results from a quasi-experimental feasibility pilot study. BMC Geriatr. 2018;18(1):204. https://doi.org/10.1186/s12877-018-0895-z .

Garcia DO, Valdez LA, Bell ML, et al. A gender- and culturally-sensitive weight loss intervention for Hispanic males: the ANIMO randomized controlled trial pilot study protocol and recruitment methods. Contemp Clin Trials Commun. 2018;9:151–63. https://doi.org/10.1016/j.conctc.2018.01.010 .

Johannessen T, Ree E, Strømme T, Aase I, Bal R, Wiig S. Designing and pilot testing of a leadership intervention to improve quality and safety in nursing homes and home care (the SAFE-LEAD intervention). BMJ Open. 2019;9(6):e027790. https://doi.org/10.1136/bmjopen-2018-027790 .

Deady M, Johnston D, Milne D, et al. Preliminary effectiveness of a smartphone app to reduce depressive symptoms in the workplace: feasibility and acceptability study. JMIR MHealth UHealth. 2018;6(12):e11661. https://doi.org/10.2196/11661 .

Juszczyk D, Gillison F. Juicy June: a mass-participation snack-swap challenge—results from a mixed methods feasibility study. Pilot Feasibil Stud. 2018;4(1):119. https://doi.org/10.1186/s40814-018-0310-8 .

Beasley JM, Kirshner L, Wylie-Rosett J, Sevick MA, DeLuca L, Chodosh J. BRInging the Diabetes prevention program to GEriatric populations (BRIDGE): a feasibility study. Pilot Feasibil Stud. 2019;5(1):129. https://doi.org/10.1186/s40814-019-0513-7 .

Prochilo GA, Costa RJS, Hassed C, Chambers R, Molenberghs P. A 16-week aerobic exercise and mindfulness-based intervention on chronic psychosocial stress: a pilot and feasibility study. Pilot Feasibil Stud. 2021;7(1):64. https://doi.org/10.1186/s40814-020-00751-6 .

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network. https://www.equator-network.org/ . Accessed 22 Aug, 2023

National Institute of Diabetes and Digestive and Kidney Diseases. U34: Implementation Planning Cooperative Agreement .; 2023. https://www.niddk.nih.gov/research-funding/process/apply/funding-mechanisms/u34-multi-center-clinical-studies . Accessed 13 July, 2023

NIH Central Resource for Grants and Funding Information. Grants & Funding: P20 .; 2023. https://grants.nih.gov/grants/funding/ac_search_results.htm?text_curr=p20 . Accessed 13 July, 2023

UK Research and Innovation. Funding for Biomedical Research and Innovation .; 2022. https://www.ukri.org/what-we-do/developing-people-and-skills/mrc/funding-for-biomedical-research-and-innovation/pilot-projects-and-small-grants/ . Accessed 13 July, 2023

Canadian Institutes of Health Research. Health Research Training Platform Pilot .; 2022. https://cihr-irsc.gc.ca/e/52278.html . Accessed 13, 2023

National Health and Medical Research Council. Research Translation .; 2023. Accessed July 13, 2023. https://www.nhmrc.gov.au/research-policy/research-translation-and-impact

National Institute of Environmental Health Sciences. Translational Research Framework .; 2019. https://www.niehs.nih.gov/research/programs/translational/framework-details/index.cfm . Accessed 13 July, 2023

Wichman C, Smith LM, Yu F. A framework for clinical and translational research in the era of rigor and reproducibility. J Clin Transl Sci. 2021;5(1):e31. https://doi.org/10.1017/cts.2020.523 .

Lee EC, Whitehead AL, Jacques RM, Julious SA. The statistical interpretation of pilot trials: should significance thresholds be reconsidered? BMC Med Res Methodol. 2014;14(1):41. https://doi.org/10.1186/1471-2288-14-41 .

Sim J. Should treatment effects be estimated in pilot and feasibility studies? Pilot Feasibil Stud. 2019;5(1):107. https://doi.org/10.1186/s40814-019-0493-7 .

Teresi JA, Yu X, Stewart AL, Hays RD. Guidelines for designing and evaluating feasibility pilot studies. Med Care. 2022;60(1):95–103. https://doi.org/10.1097/MLR.0000000000001664 .

Moore CG, Carter RE, Nietert PJ, Stewart PW. Recommendations for planning pilot studies in clinical and translational research. Clin Transl Sci. 2011;4(5):332–7. https://doi.org/10.1111/j.1752-8062.2011.00347.x .

Beets MW, Pfledderer C, Von Klinggraeff L, Burkart S, Armstrong B. Fund behavioral science like the frameworks we endorse: the case for increased funding of preliminary studies by the National Institutes of Health. Pilot Feasibil Stud. 2022;8(1):218. https://doi.org/10.1186/s40814-022-01179-w .

Download references

Acknowledgements

The authors would like to thank all experts who provided valuable input through the Delphi process of this study.

Research reported in this abstract was supported by The National Heart, Lung, and Blood Institute of the National Institutes of Health under award number R01HL149141 (Beets), F31HL158016 (von Klinggraeff), and F32HL154530 (Burkart) as well as by the Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under award number P20GM130420 for the Research Center for Child Well-Being. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Authors and affiliations.

Department of Health Promotion and Behavioral Sciences, The University of Texas Health Science Center at Houston, School of Public Health in Austin, Austin, TX, 78701, USA

Christopher D. Pfledderer

Michael and Susan Dell Center for Healthy Living, The University of Texas Health Science Center at Houston, School of Public Health in Austin, Austin, TX, 78701, USA

Arnold School of Public Health, University of South Carolina, Columbia, SC, 29205, USA

Lauren von Klinggraeff, Sarah Burkart, Alexsandra da Silva Bandeira, James F. Thrasher, Xiaoming Li & Michael W. Beets

College of Human and Social Futures, The University of Newcastle Australia, Callaghan, NSW, 2308, Australia

David R. Lubans

Bristol Medical School, Population Health Sciences, University of Bristol, Bristol, BS8 1QU, UK

Russell Jago

Faculty of Arts, Social Sciences and Humanities, School of Health and Society, University of Wollongong, Wollongong, NSW, 2522, Australia

Anthony D. Okely

MRC Epidemiology Unit, University of Cambridge, Cambridge, CB2 0QQ, UK

Esther M. F. van Sluijs

Department of Medicine, Stanford University, Stanford, CA, USA

John P. A. Ioannidis

Department of Epidemiology and Population Health, Stanford University, Stanford, CA, USA

Department of Biomedical Data Science, Stanford University, Stanford, CA, USA

Department of Statistics, Stanford University, Stanford, CA, USA

Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA

You can also search for this author in PubMed   Google Scholar

Contributions

CDP - Conceptualization; methodology; software; formal analysis; investigation; data curation; writing—original draft; writing—review and editing; visualization; and supervision. LV—Methodology; investigation; data curation; writing—original draft; writing—review and editing; and formal analysis. SB—Methodology; investigation; data curation; writing—original draft; writing—review and editing; and formal analysis. AB—Methodology; writing—original draft; writing—review and editing; and formal analysis. DL—Writing—original draft and writing—review and editing. RJ—Writing—original draft and writing—review and editing. AO—Writing—original draft and writing—review and editing. ES—Writing—original draft and writing—review and editing. JPA—Writing—original draft; writing—review and editing; and formal analysis. JT—Writing—original draft and writing—review and editing. XL—Writing—original draft and writing—review and editing. MWB—Conceptualization; methodology; software; formal analysis; investigation; data curation; writing—original draft; writing—review and editing; visualization; supervision; and funding acquisition. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Christopher D. Pfledderer .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval was granted by the University of South Carolina’s Institutional Review Board (IRB # Pro00120890) prior to the start of the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Literature crosswalk.

Additional file 2.

Summary table of considerations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Pfledderer, C.D., von Klinggraeff, L., Burkart, S. et al. Consolidated guidance for behavioral intervention pilot and feasibility studies. Pilot Feasibility Stud 10 , 57 (2024). https://doi.org/10.1186/s40814-024-01485-5

Download citation

Received : 20 September 2023

Accepted : 26 March 2024

Published : 06 April 2024

DOI : https://doi.org/10.1186/s40814-024-01485-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Pilot and Feasibility Studies

ISSN: 2055-5784

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Book of Abstract Global Mission Health Conference

• Other Journals
• For Readers
• For Authors
• For Librarians
• Steering Committe
• Keynote Speakers

LITERATURE REVIEW: PEER GROUP SUPPORT FOR SELF-CARE MANAGEMENT OF PEOPLE WITH DIABETES ME

Diabetes mellitus is a chronic disease that can cause other complications if not managed properly. What can be done to reduce the risk of complications is to implement self-care management. To improve selfcare management behavior, a strategy is needed, one of which is the provision of effective interventions to improve self-care management behavior, namely peer group support. This study aimed to determine peer group support interventions in people with diabetes mellitus in self-care management. A literature review using thematic analysis method: a simple approach. Search data sources through PubMed, Ebsco, ProQuest, and Google Scholar databases. Articles were analyzed using a Joanna Briggs Institute quasi-experimental study checklist and randomized controlled trials. Articles with journal inclusion criteria published in 2012- 2022, primary sources, with a population of people with diabetes mellitus, there is an influence of peer group support on the implementation of self-care management, there is a peer group support strategy, Indonesian and English. The keywords used were boolean "AND" and "OR". There were 1089 articles identified and published from 2012-2022. Nine articles were found that met the inclusion and exclusion criteria. Providing peer group support interventions is effective in improving self-care management of people with diabetes mellitus.

Keywords: peer group support, diabetes mellitus, self-care management

• There are currently no refbacks.

• Open access
• Published: 17 February 2022

Effectiveness of problem-based learning methodology in undergraduate medical education: a scoping review

• Joan Carles Trullàs   ORCID: orcid.org/0000-0002-7380-3475 1 , 2 , 3 ,
• Carles Blay   ORCID: orcid.org/0000-0003-3962-5887 1 , 4 ,
• Elisabet Sarri   ORCID: orcid.org/0000-0002-2435-399X 3 &
• Ramon Pujol   ORCID: orcid.org/0000-0003-2527-385X 1  

BMC Medical Education volume  22 , Article number:  104 ( 2022 ) Cite this article

31k Accesses

78 Citations

10 Altmetric

Metrics details

Problem-based learning (PBL) is a pedagogical approach that shifts the role of the teacher to the student (student-centered) and is based on self-directed learning. Although PBL has been adopted in undergraduate and postgraduate medical education, the effectiveness of the method is still under discussion. The author’s purpose was to appraise available international evidence concerning to the effectiveness and usefulness of PBL methodology in undergraduate medical teaching programs.

The authors applied the Arksey and O’Malley framework to undertake a scoping review. The search was carried out in February 2021 in PubMed and Web of Science including all publications in English and Spanish with no limits on publication date, study design or country of origin.

The literature search identified one hundred and twenty-four publications eligible for this review. Despite the fact that this review included many studies, their design was heterogeneous and only a few provided a high scientific evidence methodology (randomized design and/or systematic reviews with meta-analysis). Furthermore, most were single-center experiences with small sample size and there were no large multi-center studies. PBL methodology obtained a high level of satisfaction, especially among students. It was more effective than other more traditional (or lecture-based methods) at improving social and communication skills, problem-solving and self-learning skills. Knowledge retention and academic performance weren’t worse (and in many studies were better) than with traditional methods. PBL was not universally widespread, probably because requires greater human resources and continuous training for its implementation.

PBL is an effective and satisfactory methodology for medical education. It is likely that through PBL medical students will not only acquire knowledge but also other competencies that are needed in medical professionalism.

Peer Review reports

There has always been enormous interest in identifying the best learning methods. In the mid-twentieth century, US educator Edgar Dale proposed which actions would lead to deeper learning than others and published the well-known (and at the same time controversial) “Cone of Experience or Cone of Dale”. At the apex of the cone are oral representations (verbal descriptions, written descriptions, etc.) and at the base is direct experience (based on a person carrying out the activity that they aim to learn), which represents the greatest depth of our learning. In other words, each level of the cone corresponds to various learning methods. At the base are the most effective, participative methods (what we do and what we say) and at the apex are the least effective, abstract methods (what we read and what we hear) [ 1 ]. In 1990, psychologist George Miller proposed a framework pyramid to assess clinical competence. At the lowest level of the pyramid is knowledge (knows), followed by the competence (knows how), execution (shows how) and finally the action (does) [ 2 ]. Both Miller’s pyramid and Dale’s cone propose a very efficient way of training and, at the same time, of evaluation. Miller suggested that the learning curve passes through various levels, from the acquisition of theoretical knowledge to knowing how to put this knowledge into practice and demonstrate it. Dale stated that to remember a high percentage of the acquired knowledge, a theatrical representation should be carried out or real experiences should be simulated. It is difficult to situate methodologies such as problem-based learning (PBL), case-based learning (CBL) and team-based learning (TBL) in the context of these learning frameworks.

In the last 50 years, various university education models have emerged and have attempted to reconcile teaching with learning, according to the principle that students should lead their own learning process. Perhaps one of the most successful models is PBL that came out of the English-speaking environment. There are many descriptions of PBL in the literature, but in practice there is great variability in what people understand by this methodology. The original conception of PBL as an educational strategy in medicine was initiated at McMaster University (Canada) in 1969, leaving aside the traditional methodology (which is often based on lectures) and introducing student-centered learning. The new formulation of medical education proposed by McMaster did not separate the basic sciences from the clinical sciences, and partially abandoned theoretical classes, which were taught after the presentation of the problem. In its original version, PBL is a methodology in which the starting point is a problem or a problematic situation. The situation enables students to develop a hypothesis and identify learning needs so that they can better understand the problem and meet the established learning objectives [ 3 , 4 ]. PBL is taught using small groups (usually around 8–10 students) with a tutor. The aim of the group sessions is to identify a problem or scenario, define the key concepts identified, brainstorm ideas and discuss key learning objectives, research these and share this information with each other at subsequent sessions. Tutors are used to guide students, so they stay on track with the learning objectives of the task. Contemporary medical education also employs other small group learning methods including CBL and TBL. Characteristics common to the pedagogy of both CBL and TBL include the use of an authentic clinical case, active small-group learning, activation of existing knowledge and application of newly acquired knowledge. In CBL students are encouraged to engage in peer learning and apply new knowledge to these authentic clinical problems under the guidance of a facilitator. CBL encourages a structured and critical approach to clinical problem-solving, and, in contrast to PBL, is designed to allow the facilitator to correct and redirect students [ 5 ]. On the other hand, TBL offers a student-centered, instructional approach for large classes of students who are divided into small teams of typically five to seven students to solve clinically relevant problems. The overall similarities between PBL and TBL relate to the use of professionally relevant problems and small group learning, while the main difference relates to one teacher facilitating interactions between multiple self-managed teams in TBL, whereas each small group in PBL is facilitated by one teacher. Further differences are related to mandatory pre-reading assignments in TBL, testing of prior knowledge in TBL and activating prior knowledge in PBL, teacher-initiated clarifying of concepts that students struggled with in TBL versus students-generated issues that need further study in PBL, inter-team discussions in TBL and structured feedback and problems with related questions in TBL [ 6 ].

In the present study we have focused on PBL methodology, and, as attractive as the method may seem, we should consider whether it is really useful and effective as a learning method. Although PBL has been adopted in undergraduate and postgraduate medical education, the effectiveness (in terms of academic performance and/or skill improvement) of the method is still under discussion. This is due partly to the methodological difficulty in comparing PBL with traditional curricula based on lectures. To our knowledge, there is no systematic scoping review in the literature that has analyzed these aspects.

The main motivation for carrying out this research and writing this article was scientific but also professional interest. We believe that reviewing the state of the art of this methodology once it was already underway in our young Faculty of Medicine, could allow us to know if we were on the right track and if we should implement changes in the training of future doctors.

The primary goal of this study was to appraise available international evidence concerning to the effectiveness and usefulness of PBL methodology in undergraduate medical teaching programs. As the intention was to synthesize the scattered evidence available, the option was to conduct a scoping review. A scoping study tends to address broader topics where many different study designs might be applicable. Scoping studies may be particularly relevant to disciplines, such as medical education, in which the paucity of randomized controlled trials makes it difficult for researchers to undertake systematic reviews [ 7 , 8 ]. Even though the scoping review methodology is not widely used in medical education, it is well established for synthesizing heterogeneous research evidence [ 9 ].

The specific aims were: 1) to determine the effectiveness of PBL in academic performance (learning and retention of knowledge) in medical education; 2) to determine the effectiveness of PBL in other skills (social and communication skills, problem solving or self-learning) in medical education; 3) to know the level of satisfaction perceived by the medical students (and/or tutors) when they are taught with the PBL methodology (or when they teach in case of tutors).

This review was guided by Arksey and O’Malley’s methodological framework for conducting scoping reviews. The five main stages of the framework are: (1) identifying the research question; (2) ascertaining relevant studies; (3) determining study selection; (4) charting the data; and (5) collating, summarizing and reporting the results [ 7 ]. We reported our process according to the PRISMA Extension for Scoping Reviews [ 10 ].

Stage 1: Identifying the research question

With the goals of the study established, the four members of the research team established the research questions. The primary research question was “What is the effectiveness of PBL methodology for learning in undergraduate medicine?” and the secondary question “What is the perception and satisfaction of medical students and tutors in relation to PBL methodology?”.

Stage 2: Identifying relevant studies

After the research questions and a search strategy were defined, the searches were conducted in PubMed and Web of Science using the MeSH terms “problem-based learning” and “Medicine” (the Boolean operator “AND” was applied to the search terms). No limits were set on language, publication date, study design or country of origin. The search was carried out on 14th February 2021. Citations were uploaded to the reference manager software Mendeley Desktop (version 1.19.8) for title and abstract screening, and data characterization.

Stage 3: Study selection

The searching strategy in our scoping study generated a total of 2399 references. The literature search and screening of title, abstract and full text for suitability was performed independently by one author (JCT) based on predetermined inclusion criteria. The inclusion criteria were: 1) PBL methodology was the major research topic; 2) participants were undergraduate medical students or tutors; 3) the main outcome was academic performance (learning and knowledge retention); 4) the secondary outcomes were one of the following: social and communication skills, problem solving or self-learning and/or student/tutor satisfaction; 5) all types of studies were included including descriptive papers, qualitative, quantitative and mixed studies methods, perspectives, opinion, commentary pieces and editorials. Exclusion criteria were studies including other types of participants such as postgraduate medical students, residents and other health non-medical specialties such as pharmacy, veterinary, dentistry or nursing. Studies published in languages other than Spanish and English were also excluded. Situations in which uncertainty arose, all authors (CB, ES, RP) discussed the publication together to reach a final consensus. The outcomes of the search results and screening are presented in Fig.  1 . One-hundred and twenty-four articles met the inclusion criteria and were included in the final analysis.

Study flow PRISMA diagram. Details the review process through the different stages of the review; includes the number of records identified, included and excluded

Stage 4: Charting the data

A data extraction table was developed by the research team. Data extracted from each of the 124 publications included general publication details (year, author, and country), sample size, study population, design/methodology, main and secondary outcomes and relevant results and/or conclusions. We compiled all data into a single spreadsheet in Microsoft Excel for coding and analysis. The characteristics and the study subject of the 124 articles included in this review are summarized in Tables 1 and 2 . The detailed results of the Microsoft Excel file is also available in Additional file 1 .

Stage 5: Collating, summarizing and reporting the results

As indicated in the search strategy (Fig.  1 ) this review resulted in the inclusion of 124 publications. Publication years of the final sample ranged from 1990 to 2020, the majority of the publications (51, 41%) were identified for the years 2010–2020 and the years in which there were more publications were 2001, 2009 and 2015. Countries from the six continents were represented in this review. Most of the publications were from Asia (especially China and Saudi Arabia) and North America followed by Europe, and few studies were from Africa, Oceania and South America. The country with more publications was the United States of America ( n  = 27). The most frequent designs of the selected studies were surveys or questionnaires ( n  = 45) and comparative studies ( n  = 48, only 16 were randomized) with traditional or lecture-based learning methodologies (in two studies the comparison was with simulation) and the most frequently measured outcomes were academic performance followed by student satisfaction (48 studies measured more than one outcome). The few studies with the highest level of scientific evidence (systematic review and meta-analysis and randomized studies) were conducted mostly in Asian countries (Tables  1 and 2 ). The study subject was specified in 81 publications finding a high variability but at the same time great representability of almost all disciplines of the medical studies.

The sample size was available in 99 publications and the median [range] of the participants was 132 [14–2061]. According to study population, there were more participants in the students’ focused studies (median 134 and range 16–2061) in comparison with the tutors’ studies (median 53 and range 14–494).

Finally, after reviewing in detail the measured outcomes (main and secondary) according to the study design (Table 2 and Additional file 1 ) we present a narrative overview and a synthesis of the main findings.

Main outcome: academic performance (learning and knowledge retention)

Seventy-one of the 124 publications had learning and/or knowledge retention as a measured outcome, most of them ( n  = 45) were comparative studies with traditional or lecture-based learning and 16 were randomized. These studies were varied in their methodology, were performed in different geographic zones, and normally analyzed the experience of just one education center. Most studies ( n  = 49) reported superiority of PBL in learning and knowledge acquisition [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 ] but there was no difference between traditional and PBL curriculums in another 19 studies [ 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 ]. Only three studies reported that PBL was less effective [ 79 , 80 , 81 ], two of them were randomized (in one case favoring simulation-based learning [ 80 ] and another favoring lectures [ 81 ]) and the remaining study was based on tutors’ opinion rather than real academic performance [ 79 ]. It is noteworthy that the four systematic reviews and meta-analysis included in this scoping review, all carried out in China, found that PBL was more effective than lecture-based learning in improving knowledge and other skills (clinical, problem-solving, self-learning and collaborative) [ 40 , 51 , 53 , 58 ]. Another relevant example of the superiority of the PBL method over the traditional method is the experience reported by Hoffman et al. from the University of Missouri-Columbia. The authors analyzed the impact of implementing the PBL methodology in its Faculty of Medicine and revealed an improvement in the academic results that lasted for over a decade [ 31 ].

Secondary outcomes

Social and communication skills.

We found five studies in this scoping review that focused on these outcomes and all of them described that a curriculum centered on PBL seems to instill more confidence in social and communication skills among students. Students perceived PBL positively for teamwork, communication skills and interpersonal relations [ 44 , 45 , 67 , 75 , 82 ].

Student satisfaction

Sixty publications analyzed student satisfaction with PBL methodology. The most frequent methodology were surveys or questionnaires (30 studies) followed by comparative studies with traditional or lecture-based methodology (19 studies, 7 of them were randomized). Almost all the studies (51) have shown that PBL is generally well-received [ 11 , 13 , 18 , 19 , 20 , 21 , 22 , 26 , 29 , 34 , 37 , 39 , 41 , 42 , 46 , 50 , 56 , 58 , 63 , 64 , 66 , 78 , 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , 100 , 101 , 102 , 103 , 104 , 105 , 106 , 107 , 108 , 109 , 110 ] but in 9 studies the overall satisfaction scores for the PBL program were neutral [ 76 , 111 , 112 , 113 , 114 , 115 , 116 ] or negative [ 117 , 118 ]. Some factors that have been identified as key components for PBL to be successful include: a small group size, the use of scenarios of realistic cases and good management of group dynamics. Despite a mostly positive assessment of the PBL methodology by the students, there were some negative aspects that could be criticized or improved. These include unclear communication of the learning methodology, objectives and assessment method; bad management and organization of the sessions; tutors having little experience of the method; and a lack of standardization in the implementation of the method by the tutors.

Tutor satisfaction

There are only 15 publications that analyze the satisfaction of tutors, most of them surveys or questionnaires [ 85 , 88 , 92 , 98 , 108 , 110 , 119 ]. In comparison with the satisfaction of the students, here the results are more neutral [ 112 , 113 , 115 , 120 , 121 ] and even unfavorable to the PBL methodology in two publications [ 117 , 122 ]. PBL teaching was favored by tutors when the institutions train them in the subject, when there was administrative support and adequate infrastructure and coordination [ 123 ]. In some experiences, the PBL modules created an unacceptable toll of anxiety, unhappiness and strained relations.

Other skills (problem solving and self-learning)

The effectiveness of the PBL methodology has also been explored in other outcomes such as the ability to solve problems and to self-directed learning. All studies have shown that PBL is more effective than lecture-based learning in problem-solving and self-learning skills [ 18 , 24 , 40 , 48 , 67 , 75 , 93 , 104 , 124 ]. One single study found a poor accuracy of the students’ self-assessment when compared to their own performance [ 125 ]. In addition, there are studies that support PBL methodology for integration between basic and clinical sciences [ 126 ].

Finally, other publications have reported the experience of some faculties in the implementation of the PBL methodology. Different experiences have demonstrated that it is both possible and feasible to shift from a traditional curriculum to a PBL program, recognizing that PBL methodology is complex to plan and structure, needs a large number of human and material resources, requiring an immense teacher effort [ 28 , 31 , 94 , 127 , 128 , 129 , 130 , 131 , 132 , 133 ]. In addition, and despite its cost implication, a PBL curriculum can be successfully implemented in resource-constrained settings [ 134 , 135 ].

We conducted this scoping review to explore the effectiveness and satisfaction of PBL methodology for teaching in undergraduate medicine and, to our knowledge, it is the only study of its kind (systematic scoping review) that has been carried out in the last years. Similarly, Vernon et al. conducted a meta-analysis of articles published between 1970 and 1992 and their results generally supported the superiority of the PBL approach over more traditional methods of medical education [ 136 ]. PBL methodology is implemented in medical studies on the six continents but there is more experience (or at least more publications) from Asian countries and North America. Despite its apparent difficulties on implementation, a PBL curriculum can be successfully implemented in resource-constrained settings [ 134 , 135 ]. Although it is true that the few studies with the highest level of scientific evidence (randomized studies and meta-analysis) were carried out mainly in Asian countries (and some in North America and Europe), there were no significant differences in the main results according to geographical origin.

In this scoping review we have included a large number of publications that, despite their heterogeneity, tend to show favorable results for the usefulness of the PBL methodology in teaching and learning medicine. The results tend to be especially favorable to PBL methodology when it is compared with traditional or lecture-based teaching methods, but when compared with simulation it is not so clear. There are two studies that show neutral [ 71 ] or superior [ 80 ] results to simulation for the acquisition of specific clinical skills. It seems important to highlight that the four meta-analysis included in this review, which included a high number of participants, show results that are clearly favorable to the PBL methodology in terms of knowledge, clinical skills, problem-solving, self-learning and satisfaction [ 40 , 51 , 53 , 58 ].

Regarding the level of satisfaction described in the surveys or questionnaires, the overall satisfaction rate was higher in the PBL students when compared with traditional learning students. Students work in small groups, allowing and promoting teamwork and facilitating social and communication skills. As sessions are more attractive and dynamic than traditional classes, this could lead to a greater degree of motivation for learning.

These satisfaction results are not so favorable when tutors are asked and this may be due to different reasons; first, some studies are from the 90s, when the methodology was not yet fully implemented; second, the number of tutors included in these studies is low; and third, and perhaps most importantly, the complaints are not usually due to the methodology itself, but rather due to lack of administrative support, and/or work overload. PBL methodology implies more human and material resources. The lack of experience in guided self-learning by lecturers requires more training. Some teachers may not feel comfortable with the method and therefore do not apply it correctly.

Despite how effective and/or attractive the PBL methodology may seem, some (not many) authors are clearly detractors and have published opinion articles with fierce criticism to this methodology. Some of the arguments against are as follows: clinical problem solving is the wrong task for preclinical medical students, self-directed learning interpreted as self-teaching is not appropriate in undergraduate medical education, relegation to the role of facilitators is a misuse of the faculty, small-group experience is inherently variable and sometimes dysfunctional, etc. [ 137 ].

In light of the results found in our study, we believe that PBL is an adequate methodology for the training of future doctors and reinforces the idea that the PBL should have an important weight in the curriculum of our medical school. It is likely that training through PBL, the doctors of the future will not only have great knowledge but may also acquire greater capacity for communication, problem solving and self-learning, all of which are characteristics that are required in medical professionalism. For this purpose, Koh et al. analyzed the effect that PBL during medical school had on physician competencies after graduation, finding a positive effect mainly in social and cognitive dimensions [ 138 ].

Despite its defects and limitations, we must not abandon this methodology and, in any case, perhaps PBL should evolve, adapt, and improve to enhance its strengths and improve its weaknesses. It is likely that the new generations, trained in schools using new technologies and methodologies far from lectures, will feel more comfortable (either as students or as tutors) with methodologies more like PBL (small groups and work focused on problems or projects). It would be interesting to examine the implementation of technologies and even social media into PBL sessions, an issue that has been poorly explorer [ 139 ].

Limitations

Scoping reviews are not without limitations. Our review includes 124 articles from the 2399 initially identified and despite our efforts to be as comprehensive as possible, we may have missed some (probably few) articles. Even though this review includes many studies, their design is very heterogeneous, only a few include a large sample size and high scientific evidence methodology. Furthermore, most are single-center experiences and there are no large multi-center studies. Finally, the frequency of the PBL sessions (from once or twice a year to the whole curriculum) was not considered, in part, because most of the revised studies did not specify this information. This factor could affect the efficiency of PBL and the perceptions of students and tutors about PBL. However, the adoption of a scoping review methodology was effective in terms of summarizing the research findings, identifying limitations in studies’ methodologies and findings and provided a more rigorous vision of the international state of the art.

Conclusions

This systematic scoping review provides a broad overview of the efficacy of PBL methodology in undergraduate medicine teaching from different countries and institutions. PBL is not a new teaching method given that it has already been 50 years since it was implemented in medicine courses. It is a method that shifts the leading role from teachers to students and is based on guided self-learning. If it is applied properly, the degree of satisfaction is high, especially for students. PBL is more effective than traditional methods (based mainly on lectures) at improving social and communication skills, problem-solving and self-learning skills, and has no worse results (and in many studies better results) in relation to academic performance. Despite that, its use is not universally widespread, probably because it requires greater human resources and continuous training for its implementation. In any case, more comparative and randomized studies and/or other systematic reviews and meta-analysis are required to determine which educational strategies could be most suitable for the training of future doctors.

Abbreviations

• Problem-based learning

Case-based learning

Team-based learning

References:

Dale E. Methods for analyzing the content of motion pictures. J Educ Sociol. 1932;6:244–50.

Google Scholar  

Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9 Suppl):S63–7. https://doi.org/10.1097/00001888-199009000-00045 .

Article   Google Scholar  

Bodagh N, Bloomfield J, Birch P, Ricketts W. Problem-based learning: a review. Br J Hosp Med (Lond). 2017;78:C167–70. https://doi.org/10.12968/hmed.2017.78.11.C167 .

- Branda LA. El abc del ABP: Lo esencial del aprendizaje basado en problemas. In: Fundación Dr. Esteve, Cuadernos de la fundación Dr. Antonio Esteve nº27: El aprendizaje basado en problemas en sus textos, pp.1–16. 2013. Barcelona.

Burgess A, Matar E, Roberts C, et al. Scaffolding medical student knowledge and skills: team-based learning (TBL) and case-based learning (CBL). BMC Med Educ. 2021;21:238. https://doi.org/10.1186/s12909-021-02638-3 .

Dolmans D, Michaelsen L, van Merriënboer J, van der Vleuten C. Should we choose between problem-based learning and team-based learning? No, combine the best of both worlds! Med Teach. 2015;37:354–9. https://doi.org/10.3109/0142159X.2014.948828 .

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. In J Soc Res Methodol. 2005;8:19–32. https://doi.org/10.1080/1364557032000119616 .

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69. https://doi.org/10.1186/1748-5908-5-69 .

Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5:371–85. https://doi.org/10.1002/jrsm.1123 .

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467–73. https://doi.org/10.7326/M18-0850 .

Sokas RK, Diserens D, Johnston MA. Integrating occupational-health into the internal medicine clerkship using problem-based learning. Clin Res. 1990;38:A735.

Richards BF, Ober KP, Cariaga-Lo L, et al. Ratings of students’ performances in a third-year internal medicine clerkship: a comparison between problem-based and lecture-based curricula. Acad Med. 1996;71:187–9. https://doi.org/10.1097/00001888-199602000-00028 .

Gresham CL, Philp JR. Problem-based learning in clinical medicine. Teach Learn Med. 1996;8:111–5. https://doi.org/10.1080/10401339609539776 .

Hill J, Rolfe IE, Pearson SA, Heathcote A. Do junior doctors feel they are prepared for hospital practice? A study of graduates from traditional and non-traditional medical schools. Med Educ. 1998;32:19–24. https://doi.org/10.1046/j.1365-2923.1998.00152.x .

Blake RL, Parkison L. Faculty evaluation of the clinical performances of students in a problem-based learning curriculum. Teach Learn Med. 1998;10:69–73. https://doi.org/10.1207/S15328015TLM1002\_3 .

Hmelo CE. Problem-based learning: effects on the early acquisition of cognitive skill in medicine. J Learn Sc. 1998;7:173–208. https://doi.org/10.1207/s15327809jls0702\_2 .

Finch PN. The effect of problem-based learning on the academic performance of students studying podiatric medicine in Ontario. Med Educ. 1999;33:411–7.

Casassus P, Hivon R, Gagnayre R, d’Ivernois JF. An initial experiment in haematology instruction using the problem-based learning method in third-year medical training in France. Hematol Cell Ther. 1999;41:137–44. https://doi.org/10.1007/s00282-999-0137-0 .

Purdy RA, Benstead TJ, Holmes DB, Kaufman DM. Using problem-based learning in neurosciences education for medical students. Can J Neurol Sci. 1999;26:211–6. https://doi.org/10.1017/S0317167100000287 .

Farrell TA, Albanese MA, Pomrehn PRJ. Problem-based learning in ophthalmology: a pilot program for curricular renewal. Arch Ophthalmol. 1999;117:1223–6. https://doi.org/10.1001/archopht.117.9.1223 .

Curtis JA, Indyk D, Taylor B. Successful use of problem-based learning in a third-year pediatric clerkship. Ambul Pediatr. 2001;1:132–5. https://doi.org/10.1367/1539-4409(2001)001%3c0132:suopbl%3e2.0.co;2 .

Trevena LJ, Clarke RM. Self-directed learning in population health. a clinically relevant approach for medical students. Am J Prev Med. 2002;22:59–65. https://doi.org/10.1016/s0749-3797(01)00395-6 .

Astin J, Jenkins T, Moore L. Medical students’ perspective on the teaching of medical statistics in the undergraduate medical curriculum. Stat Med. 2002;21:1003–7. https://doi.org/10.1002/sim.1132 .

Whitfield CR, Manger EA, Zwicker J, Lehman EB. Differences between students in problem-based and lecture-based curricula measured by clerkship performance ratings at the beginning of the third year. Teach Learn Med. 2002;14:211–7. https://doi.org/10.1207/S15328015TLM1404\_2 .

McParland M, Noble LM, Livingston G. The effectiveness of problem-based learning compared to traditional teaching in undergraduate psychiatry. Med Educ. 2004;38:859–67. https://doi.org/10.1111/j.1365-2929.2004.01818.x .

Casey PM, Magrane D, Lesnick TG. Improved performance and student satisfaction after implementation of a problem-based preclinical obstetrics and gynecology curriculum. Am J Obstet Gynecol. 2005;193:1874–8. https://doi.org/10.1016/j.ajog.2005.07.061 .

Gurpinar E, Musal B, Aksakoglu G, Ucku R. Comparison of knowledge scores of medical students in problem-based learning and traditional curriculum on public health topics. BMC Med Educ. 2005;5:7. https://doi.org/10.1186/1472-6920-5-7 .

Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Effect of a community oriented problem based learning curriculum on quality of primary care delivered by graduates: historical cohort comparison study. BMJ. 2005;331:1002. https://doi.org/10.1136/bmj.38636.582546.7C .

Abu-Hijleh MF, Chakravarty M, Al-Shboul Q, Kassab S, Hamdy H. Integrating applied anatomy in surgical clerkship in a problem-based learning curriculum. Surg Radiol Anat. 2005;27:152–7. https://doi.org/10.1007/s00276-004-0293-4 .

Distlehorst LH, Dawson E, Robbs RS, Barrows HS. Problem-based learning outcomes: the glass half-full. Acad Med. 2005;80:294–9. https://doi.org/10.1097/00001888-200503000-00020 .

Hoffman K, Hosokawa M, Blake R Jr, Headrick L, Johnson G. Problem-based learning outcomes: ten years of experience at the University of Missouri-Columbia school of medicine. Acad Med. 2006;81:617–25. https://doi.org/10.1097/01.ACM.0000232411.97399.c6 .

Kong J, Li X, Wang Y, Sun W, Zhang J. Effect of digital problem-based learning cases on student learning outcomes in ophthalmology courses. Arch Ophthalmol. 2009;127:1211–4. https://doi.org/10.1001/archophthalmol.2009.110 .

Tsou KI, Cho SL, Lin CS, et al. Short-term outcomes of a near-full PBL curriculum in a new Taiwan medical school. Kaohsiung J Med Sci. 2009;25:282–93. https://doi.org/10.1016/S1607-551X(09)70075-0 .

Wang J, Zhang W, Qin L, et al. Problem-based learning in regional anatomy education at Peking University. Anat Sci Educ. 2010;3:121–6. https://doi.org/10.1002/ase.151 .

Abou-Elhamd KA, Rashad UM, Al-Sultan AI. Applying problem-based learning to otolaryngology teaching. J Laryngol Otol. 2011;125:117–20. https://doi.org/10.1017/S0022215110001702 .

Urrutia Aguilar ME, Hamui-Sutton A, Castaneda Figueiras S, van der Goes TI, Guevara-Guzman R. Impact of problem-based learning on the cognitive processes of medical students. Gac Med Mex. 2011;147:385–93.

Tian J-H, Yang K-H, Liu A-P. Problem-based learning in evidence-based medicine courses at Lanzhou University. Med Teach. 2012;34:341. https://doi.org/10.3109/0142159X.2011.531169 .

Hoover CR, Wong CC, Azzam A. From primary care to public health: using problem-based Learning and the ecological model to teach public health to first year medical students. J Community Health. 2012;37:647–52. https://doi.org/10.1007/s10900-011-9495-y .

Li J, Li QL, Li J, et al. Comparison of three problem-based learning conditions (real patients, digital and paper) with lecture-based learning in a dermatology course: a prospective randomized study from China. Med Teach. 2013;35:e963–70. https://doi.org/10.3109/0142159X.2012.719651 .

Ding X, Zhao L, Chu H, et al. Assessing the effectiveness of problem-based learning of preventive medicine education in China. Sci Rep. 2014;4:5126. https://doi.org/10.1038/srep05126 .

Meo SA. Undergraduate medical student’s perceptions on traditional and problem based curricula: pilot study. J Pak Med Assoc. 2014;64:775–9.

Khoshnevisasl P, Sadeghzadeh M, Mazloomzadeh S, Hashemi Feshareki R, Ahmadiafshar A. Comparison of problem-based learning with lecture-based learning. Iran Red Crescent Med J. 2014;16: e5186. https://doi.org/10.5812/ircmj.5186 .

Al-Drees AA, Khalil MS, Irshad M, Abdulghani HM. Students’ perception towards the problem based learning tutorial session in a system-based hybrid curriculum. Saudi Med J. 2015;36:341–8. https://doi.org/10.15537/smj.2015.3.10216 .

Al-Shaikh G, Al Mussaed EM, Altamimi TN, Elmorshedy H, Syed S, Habib F. Perception of medical students regarding problem based learning. Kuwait Med J. 2015;47:133–8.

Hande S, Mohammed CA, Komattil R. Acquisition of knowledge, generic skills and attitudes through problem-based learning: student perspectives in a hybrid curriculum. J Taibah Univ Medical Sci. 2015;10:21–5. https://doi.org/10.1016/j.jtumed.2014.01.008 .

González Mirasol E, Gómez García MT, Lobo Abascal P, Moreno Selva R, Fuentes Rozalén AM, González MG. Analysis of perception of training in graduates of the faculty of medicine at Universidad de Castilla-Mancha. Eval Program Plann. 2015;52:169–75. https://doi.org/10.1016/j.evalprogplan.2015.06.001 .

Yanamadala M, Kaprielian VS, O’Connor Grochowski C, Reed T, Heflin MT. A problem-based learning curriculum in geriatrics for medical students. Gerontol Geriatr Educ. 2018;39:122–31. https://doi.org/10.1080/02701960.2016.1152268 .

Balendran K, John L. Comparison of learning outcomes in problem based learning and lecture based learning in teaching forensic medicine. J Evol Med Dent Sci. 2017;6:89–92. https://doi.org/10.14260/jemds/2017/22 .

Chang H-C, Wang N-Y, Ko W-R, Yu Y-T, Lin L-Y, Tsai H-F. The effectiveness of clinical problem-based learning model of medico-jurisprudence education on general law knowledge for obstetrics/gynecological interns. Taiwan J Obstet Gynecol. 2017;56:325–30. https://doi.org/10.1016/j.tjog.2017.04.011 .

Eltony SA, El-Sayed NH, El-Araby SE-S, Kassab SE. Implementation and evaluation of a patient safety course in a problem-based learning program. Educ Heal. 2017;30:44–9. https://doi.org/10.4103/1357-6283.210512 .

Zhang S, Xu J, Wang H, Zhang D, Zhang Q, Zou L. Effects of problem-based learning in Chinese radiology education: a systematic review and meta-analysis. Medicine (Baltimore). 2018;97: e0069. https://doi.org/10.1097/MD.0000000000010069 .

Hincapie Parra DA, Ramos Monobe A, Chrino-Barcelo V. Problem based learning as an active learning strategy and its impact on academic performance and critical thinking of medical students. Rev Complut Educ. 2018;29:665–81. https://doi.org/10.5209/RCED.53581 .

Ma Y, Lu X. The effectiveness of problem-based learning in pediatric medical education in China: a meta-analysis of randomized controlled trials. Medicine (Baltimore). 2019;98: e14052. https://doi.org/10.1097/MD.0000000000014052 .

Berger C, Brinkrolf P, Ertmer C, et al. Combination of problem-based learning with high-fidelity simulation in CPR training improves short and long-term CPR skills: a randomised single blinded trial. BMC Med Educ. 2019;19:180. https://doi.org/10.1186/s12909-019-1626-7 .

Aboonq M, Alquliti A, Abdulmonem I, Alpuq N, Jalali K, Arabi S. Students’ approaches to learning and perception of learning environment: a comparison between traditional and problem-based learning medical curricula. Indo Am J Pharm Sci. 2019;6:3610–9. https://doi.org/10.5281/zenodo.2562660 .

Li X, Xie F, Li X, et al. Development, application, and evaluation of a problem-based learning method in clinical laboratory education. Clin Chim ACTA. 2020;510:681–4. https://doi.org/10.1016/j.cca.2020.08.037 .

Zhao W, He L, Deng W, Zhu J, Su A, Zhang Y. The effectiveness of the combined problem-based learning (PBL) and case-based learning (CBL) teaching method in the clinical practical teaching of thyroid disease. BMC Med Educ. 2020;20:381. https://doi.org/10.1186/s12909-020-02306 .

Liu C-X, Ouyang W-W, Wang X-W, Chen D, Jiang Z-L. Comparing hybrid problem-based and lecture learning (PBL plus LBL) with LBL pedagogy on clinical curriculum learning for medical students in China: a meta-analysis of randomized controlled trials. Medicine (Baltimore). 2020;99:e19687. https://doi.org/10.1097/MD.0000000000019687 .

Margolius SW, Papp KK, Altose MD, Wilson-Delfosse AL. Students perceive skills learned in pre-clerkship PBL valuable in core clinical rotations. Med Teach. 2020;42:902–8. https://doi.org/10.1080/0142159X.2020.1762031 .

Schwartz RW, Donnelly MB, Nash PP, Young B. Developing students cognitive skills in a problem-based surgery clerkship. Acad Med. 1992;67:694–6. https://doi.org/10.1097/00001888-199210000-00016 .

Mennin SP, Friedman M, Skipper B, Kalishman S, Snyder J. Performances on the NBME-I, NBME-II, and NBME-III by medical-students in the problem-based learning and conventional tracks at the university-of-new-mexico. Acad Med. 1993;68:616–24. https://doi.org/10.1097/00001888-199308000-00012 .

Kaufman DM, Mann KV. Comparing achievement on the medical council of Canada qualifying examination part I of students in conventional and problem-based learning curricula. Acad Med. 1998;73:1211–3. https://doi.org/10.1097/00001888-199811000-00022 .

Kaufman DM, Mann KV. Achievement of students in a conventional and Problem-Based Learning (PBL) curriculum. Adv Heal Sci Educ. 1999;4:245–60. https://doi.org/10.1023/A:1009829831978 .

Antepohl W, Herzig S. Problem-based learning versus lecture-based learning in a course of basic pharmacology: a controlled, randomized study. Med Educ. 1999;33:106–13. https://doi.org/10.1046/j.1365-2923.1999.00289.x .

Dyke P, Jamrozik K, Plant AJ. A randomized trial of a problem-based learning approach for teaching epidemiology. Acad Med. 2001;76:373–9. https://doi.org/10.1097/00001888-200104000-00016 .

Brewer DW. Endocrine PBL in the year 2000. Adv Physiol Educ. 2001;25:249–55. https://doi.org/10.1152/advances.2001.25.4.249 .

Seneviratne RD, Samarasekera DD, Karunathilake IM, Ponnamperuma GG. Students’ perception of problem-based learning in the medical curriculum of the faculty of medicine, University of Colombo. Ann Acad Med Singapore. 2001;30:379–81.

Alleyne T, Shirley A, Bennett C, et al. Problem-based compared with traditional methods at the faculty of medical sciences, University of the West Indies: a model study. Med Teach. 2002;24:273–9. https://doi.org/10.1080/01421590220125286 .

Norman GR, Wenghofer E, Klass D. Predicting doctor performance outcomes of curriculum interventions: problem-based learning and continuing competence. Med Educ. 2008;42:794–9. https://doi.org/10.1111/j.1365-2923.2008.03131.x .

Cohen-Schotanus J, Muijtjens AMM, Schoenrock-Adema J, Geertsma J, van der Vleuten CPM. Effects of conventional and problem-based learning on clinical and general competencies and career development. Med Educ. 2008;42:256–65. https://doi.org/10.1111/j.1365-2923.2007.02959.x .

Wenk M, Waurick R, Schotes D, et al. Simulation-based medical education is no better than problem-based discussions and induces misjudgment in self-assessment. Adv Health Sci Educ Theory Pract. 2009;14:159–71. https://doi.org/10.1007/s10459-008-9098-2 .

Collard A, Gelaes S, Vanbelle S, et al. Reasoning versus knowledge retention and ascertainment throughout a problem-based learning curriculum. Med Educ. 2009;43:854–65. https://doi.org/10.1111/j.1365-2923.2009.03410.x .

Nouns Z, Schauber S, Witt C, Kingreen H, Schuettpelz-Brauns K. Development of knowledge in basic sciences: a comparison of two medical curricula. Med Educ. 2012;46:1206–14. https://doi.org/10.1111/medu.12047 .

Saloojee S, van Wyk J. The impact of a problem-based learning curriculum on the psychiatric knowledge and skills of final-year students at the Nelson R Mandela school of medicine. South African J Psychiatry. 2012;18:116.

Mughal AM, Shaikh SH. Assessment of collaborative problem solving skills in undergraduate medical students at Ziauddin college of medicine. Karachi Pakistan J Med Sci. 2018;34:185–9. https://doi.org/10.12669/pjms.341.13485 .

Hu X, Zhang H, Song Y, et al. Implementation of flipped classroom combined with problem-based learning: an approach to promote learning about hyperthyroidism in the endocrinology internship. BMC Med Educ. 2019;19:290. https://doi.org/10.1186/s12909-019-1714-8 .

Thompson KL, Gendreau JL, Strickling JE, Young HE. Cadaveric dissection in relation to problem-based learning case sequencing: a report of medical student musculoskeletal examination performances and self-confidence. Anat Sci Educ. 2019;12:619–26. https://doi.org/10.1002/ase.1891 .

Chang G, Cook D, Maguire T, Skakun E, Yakimets WW, Warnock GL. Problem-based learning: its role in undergraduate surgical education. Can J Surg. 1995;38:13–21.

Vernon DTA, Hosokawa MC. Faculty attitudes and opinions about problem-based learning. Acad Med. 1996;71:1233–8. https://doi.org/10.1097/00001888-199611000-00020 .

Steadman RH, Coates WC, Huang YM, et al. Simulation-based training is superior to problem-based learning for the acquisition of critical assessment and management skills. Crit Care Med. 2006;34:151–7. https://doi.org/10.1097/01.CCM.0000190619.42013.94 .

Johnston JM, Schooling CM, Leung GM. A randomised-controlled trial of two educational modes for undergraduate evidence-based medicine learning in Asia. BMC Med Educ. 2009;9:63. https://doi.org/10.1186/1472-6920-9-63 .

Suleman W, Iqbal R, Alsultan A, Baig SM. Perception of 4(th) year medical students about problem based learning. Pakistan J Med Sci. 2010;26:871–4.

Blosser A, Jones B. Problem-based learning in a surgery clerkship. Med Teach. 1991;13:289–93. https://doi.org/10.3109/01421599109089907 .

Usherwood T, Joesbury H, Hannay D. Student-directed problem-based learning in general-practice and public-health medicine. Med Educ. 1991;25:421–9. https://doi.org/10.1111/j.1365-2923.1991.tb00090.x .

Bernstein P, Tipping J, Bercovitz K, Skinner HA. Shifting students and faculty to a PBL curriculum - attitudes changed and lessons learned. Acad Med. 1995;70:245–7. https://doi.org/10.1097/00001888-199503000-00019 .

Kaufman DM, Mann KV. Comparing students’ attitudes in problem-based and conventional curricula. Acad Med. 1996;71:1096–9. https://doi.org/10.1097/00001888-199610000-00018 .

Kalaian HA, Mullan PB. Exploratory factor analysis of students’ ratings of a problem-based learning curriculum. Acad Med. 1996;71:390–2. https://doi.org/10.1097/00001888-199604000-00019 .

Vincelette J, Lalande R, Delorme P, Goudreau J, Lalonde V, Jean P. A pilot course as a model for implementing a PBL curriculum. Acad Med. 1997;72:698–701. https://doi.org/10.1097/00001888-199708000-00015 .

Ghosh S, Dawka V. Combination of didactic lecture with problem-based learning sessions in physiology teaching in a developing medical college in Nepal. Adv Physiol Educ. 2000;24:8–12.

Walters MR. Problem-based learning within endocrine physiology lectures. Adv Physiol Educ. 2001;25:225–7. https://doi.org/10.1152/advances.2001.25.4.225 .

Leung GM, Lam TH, Hedley AJ. Problem-based public health learning - from the classroom to the community. Med Educ. 2001;35:1071–2.

Khoo HE, Chhem RK, Gwee MCE, Balasubramaniam P. Introduction of problem-based learning in a traditional medical curriculum in Singapore - students’ and tutors’ perspectives. Ann Acad Med Singapore. 2001;30:371–4.

Villamor MCA. Problem-based learning (PBL) as an approach in the teaching of biochemistry of the endocrine system at the Angeles University College of Medicine. Ann Acad Med Singapore. 2001;30:382–6.

Chang C-H, Yang C-Y, See L-C, Lui P-W. High satisfaction with problem-based learning for anesthesia. Chang Gung Med J. 2004;27:654–62.

McLean M. A comparison of students who chose a traditional or a problem-based learning curriculum after failing year 2 in the traditional curriculum: a unique case study at the Nelson R. Mandela school of medicine. Teach Learn Med. 2004;16:301–3. https://doi.org/10.1207/s15328015tlm1603\_15 .

Lucas M, García Guasch R, Moret E, Llasera R, Melero A. Canet J [Problem-based learning in an undergraduate medical school course on anesthesiology, recovery care, and pain management]. Rev Esp Anestesiol Reanim. 2006;53:419–25.

Burgun A, Darmoni S, Le Duff F, Weber J. Problem-based learning in medical informatics for undergraduate medical students: an experiment in two medical schools. Int J Med Inform. 2006;75:396–402. https://doi.org/10.1016/j.ijmedinf.2005.07.014 .

Gurpinar E, Senol Y, Aktekin MR. Evaluation of problem based learning by tutors and students in a medical faculty of Turkey. Kuwait Med J. 2009;41:123–7.

Elzubeir MA. Teaching of the renal system in an integrated, problem-based curriculum. Saudi J Kidney Dis Transpl. 2012;23:93–8.

Sulaiman N, Hamdy H. Problem-based learning: where are we now? Guide supplement 36.3–practical application. Med Teach. 2013;35:160–2. https://doi.org/10.3109/0142159X.2012.737965 .

Albarrak AI, Mohammed R, Abalhassan MF, Almutairi NK. Academic satisfaction among traditional and problem based learning medical students a comparative study. Saudi Med J. 2013;34:1179–88.

Nosair E, Mirghani Z, Mostafa RM. Measuring students’ perceptions of educational environment in the PBL program of Sharjah Medical College. J Med Educ Curric Dev. 2015;2:71–9. https://doi.org/10.4137/JMECDECDECD.S29926 .

Tshitenge ST, Ndhlovu CE, Ogundipe R. Evaluation of problem-based learning curriculum implementation in a clerkship rotation of a newly established African medical training institution: lessons from the University of Botswana. Pan Afr Med J. 2017;27:13. https://doi.org/10.11604/pamj.2017.27.13.10623 .

Yadav RL, Piryani RM, Deo GP, Shah DK, Yadav LK, Islam MN. Attitude and perception of undergraduate medical students toward the problem-based learning in Chitwan Medical College. Nepal Adv Med Educ Pract. 2018;9:317–22. https://doi.org/10.2147/AMEP.S160814 .

Asad MR, Tadvi N, Amir KM, Afzal K, Irfan A, Hussain SA. Medical student’s feedback towards problem based learning and interactive lectures as a teaching and learning method in an outcome-based curriculum. Int J Med Res & Heal Sci. 2019;8:78–84. https://doi.org/10.33844/ijol.2019.60392 .

Mpalanyi M, Nalweyiso ID, Mubuuke AG. Perceptions of radiography students toward problem-based learning almost two decades after its introduction at Makerere University. Uganda J Med imaging Radiat Sci. 2020;51:639–44. https://doi.org/10.1016/j.jmir.2020.06.009 .

Korkmaz NS, Ozcelik S. Evaluation of the opinions of the first, second and third term medical students about problem based learning sessions in Bezmialem Vakif University. Bezmialem Sci. 2020;8:144–9. https://doi.org/10.14235/bas.galenos.2019.3471 .

McGrew MC, Skipper B, Palley T, Kaufman A. Student and faculty perceptions of problem-based learning on a family medicine clerkship. Fam Med. 1999;31:171–6.

Kelly AM. A problem-based learning resource in emergency medicine for medical students. J Accid Emerg Med. 2000;17:320–3. https://doi.org/10.1136/emj.17.5.320 .

Bui-Mansfield LT, Chew FS. Radiologists as clinical tutors in a problem-based medical school curriculum. Acad Radiol. 2001;8:657–63. https://doi.org/10.1016/S1076-6332(03)80693-1 .

Macallan DC, Kent A, Holmes SC, Farmer EA, McCrorie P. A model of clinical problem-based learning for clinical attachments in medicine. Med Educ. 2009;43:799–807. https://doi.org/10.1111/j.1365-2923.2009.03406.x .

Grisham JW, Martiniuk ALC, Negin J, Wright EP. Problem-based learning (PBL) and public health: an initial exploration of perceptions of PBL in Vietnam. Asia-Pacific J public Heal. 2015;27:NP2019-27. https://doi.org/10.1177/1010539512436875 .

Khan IA, Al-Swailmi FK. Perceptions of faculty and students regarding Problem Based Learning: a mixed methods study. J Pak Med Assoc. 2015;65:1334–8.

Alduraywish AA, Mohager MO, Alenezi MJ, Nail AM, Aljafari AS. Evaluation of students’ experience with Problem-based Learning (PBL) applied at the College of Medicine, Al-Jouf University. Saudi Arabia J Pak Med Assoc. 2017;67:1870–3.

Yoo DM, Cho AR, Kim S. Satisfaction with and suitability of the problem-based learning program at the Catholic University of Korea College of Medicine. J Educ Eval Health Prof. 2019;16:20. https://doi.org/10.3352/jeehp.2019.16.20 .

Aldayel AA, Alali AO, Altuwaim AA, et al. Problem-based learning: medical students’ perception toward their educational environment at Al-Imam Mohammad Ibn Saud Islamic University. Adv Med Educ Pract. 2019;10:95–104. https://doi.org/10.2147/AMEP.S189062 .

DeLowerntal E. An evaluation of a module in problem-based learning. Int J Educ Dev. 1996;16:303–7. https://doi.org/10.1016/0738-0593(96)00001-6 .

Tufts MA, Higgins-Opitz SB. What makes the learning of physiology in a PBL medical curriculum challenging? Student perceptions. Adv Physiol Educ. 2009;33:187–95. https://doi.org/10.1152/advan.90214.2008 .

Aboonq M. Perception of the faculty regarding problem-based learning as an educational approach in Northwestern Saudi Arabia. Saudi Med J. 2015;36:1329–35. https://doi.org/10.15537/smj.2015.11.12263 .

Subramaniam RM, Scally P, Gibson R. Problem-based learning and medical student radiology teaching. Australas Radiol. 2004;48:335–8. https://doi.org/10.1111/j.0004-8461.2004.01317.x .

Chang BJ. Problem-based learning in medical school: a student’s perspective. Ann Med Surg. 2016;12:88–9. https://doi.org/10.1016/j.amsu.2016.11.011 .

Griffith CD, Blue AV, Mainous AG, DeSimone PA. Housestaff attitudes toward a problem-based clerkship. Med Teach. 1996;18:133–4. https://doi.org/10.3109/01421599609034147 .

Navarro HN, Zamora SJ. The opinion of teachers about tutorial problem based learning. Rev Med Chil. 2014;142:989–97. https://doi.org/10.4067/S0034-98872014000800006 .

Demiroren M, Turan S, Oztuna D. Medical students’ self-efficacy in problem-based learning and its relationship with self-regulated learning. Med Educ Online. 2016;21:30049. https://doi.org/10.3402/meo.v21.30049 .

Tousignant M, DesMarchais JE. Accuracy of student self-assessment ability compared to their own performance in a problem-based learning medical program: a correlation study. Adv Heal Sci Educ. 2002;7:19–27. https://doi.org/10.1023/A:1014516206120 .

Brynhildsen J, Dahle LO, Behrbohm Fallsberg M, Rundquist I, Hammar M. Attitudes among students and teachers on vertical integration between clinical medicine and basic science within a problem-based undergraduate medical curriculum. Med Teach. 2002;24:286–8. https://doi.org/10.1080/01421590220134105 .

Desmarchais JE. A student-centered, problem-based curriculum - 5 years experience. Can Med Assoc J. 1993;148:1567–72.

Doig K, Werner E. The marriage of a traditional lecture-based curriculum and problem-based learning: are the offspring vigorous? Med Teach. 2000;22:173–8.

Kemahli S. Hematology education in a problem-based curriculum. Hematology. 2005;10(Suppl 1):161–3. https://doi.org/10.1080/10245330512331390267 .

Grkovic I. Transition of the medical curriculum from classical to integrated: problem-based approach and Australian way of keeping academia in medicine. Croat Med J. 2005;46:16–20.

Bosch-Barrera J, Briceno Garcia HC, Capella D, et al. Teaching bioethics to students of medicine with Problem-Based Learning (PBL). Cuad Bioet. 2015;26:303–9.

Lin Y-C, Huang Y-S, Lai C-S, Yen J-H, Tsai W-C. Problem-based learning curriculum in medical education at Kaohsiung Medical University. Kaohsiung J Med Sci. 2009;25:264–9. https://doi.org/10.1016/S1607-551X(09)70072-5 .

Salinas Sánchez AS, Hernández Millán I, Virseda Rodríguez JA, et al. Problem-based learning in urology training the faculty of medicine of the Universidad de Castilla-La Mancha model. Actas Urol Esp. 2005;29:8–15. https://doi.org/10.1016/s0210-4806(05)73193-4 .

Amoako-Sakyi D, Amonoo-Kuofi H. Problem-based learning in resource-poor settings: lessons from a medical school in Ghana. BMC Med Educ. 2015;15:221. https://doi.org/10.1186/s12909-015-0501-4 .

Carrera LI, Tellez TE, D’Ottavio AE. Implementing a problem-based learning curriculum in an Argentinean medical school: implications for developing countries. Acad Med. 2003;78:798–801. https://doi.org/10.1097/00001888-200308000-00010 .

Vernon DT, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med. 1993;68:550–63. https://doi.org/10.1097/00001888-199307000-00015 .

Shanley PF. Viewpoint: leaving the “empty glass” of problem-based learning behind: new assumptions and a revised model for case study in preclinical medical education. Acad Med. 2007;82:479–85. https://doi.org/10.1097/ACM.0b013e31803eac4c .

Koh GC, Khoo HE, Wong ML, Koh D. The effects of problem-based learning during medical school on physician competency: a systematic review. CMAJ. 2008;178:34–41. https://doi.org/10.1503/cmaj.070565 .

Awan ZA, Awan AA, Alshawwa L, Tekian A, Park YS. Assisting the integration of social media in problem-based learning sessions in the faculty of medicine at King Abdulaziz University. Med Teach. 2018;40:S37–42. https://doi.org/10.1080/0142159X.2018.1465179 .

Download references

Acknowledgements

Not applicable

No funding was received for conducting this study.

Author information

Authors and affiliations.

Medical Education Cathedra, School of Medicine, University of Vic-Central University of Catalonia, Vic, Barcelona, Spain

Joan Carles Trullàs, Carles Blay & Ramon Pujol

Internal Medicine Service, Hospital de Olot i Comarcal de La Garrotxa, Olot, Girona, Spain

Joan Carles Trullàs

The Tissue Repair and Regeneration Laboratory (TR2Lab), University of Vic-Central University of Catalonia, Vic, Barcelona, Spain

Joan Carles Trullàs & Elisabet Sarri

Catalan Institute of Health (ICS) – Catalunya Central, Barcelona, Spain

Carles Blay

You can also search for this author in PubMed   Google Scholar

Contributions

JCT had the idea for the article, performed the literature search and data analysis and drafted the first version of the manuscript. CB, ES and RP contributed to the data analysis and suggested revisions to the manuscript. All authors read and approved the final manuscript.

Ethics declarations

Availability of data and materials.

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Ethics approval and consent to participate

Not applicable for a literature review.

Consent for publication

Competing interests.

All authors declare that they have no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Characteristics ofthe 124 included studies.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Trullàs, J.C., Blay, C., Sarri, E. et al. Effectiveness of problem-based learning methodology in undergraduate medical education: a scoping review. BMC Med Educ 22 , 104 (2022). https://doi.org/10.1186/s12909-022-03154-8

Download citation

Received : 03 October 2021

Accepted : 02 February 2022

Published : 17 February 2022

DOI : https://doi.org/10.1186/s12909-022-03154-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic review

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 01 April 2024

The impact of housing prices on residents’ health: a systematic review

• Ashmita Grewal 1 ,
• Kirk J. Hepburn 1 ,
• Scott A. Lear 1 ,
• Marina Adshade 2 &
• Kiffer G. Card 1  

BMC Public Health volume  24 , Article number:  931 ( 2024 ) Cite this article

430 Accesses

19 Altmetric

Metrics details

Rising housing prices are becoming a top public health priority and are an emerging concern for policy makers and community leaders. This report reviews and synthesizes evidence examining the association between changes in housing price and health outcomes.

We conducted a systematic literature review by searching the SCOPUS and PubMed databases for keywords related to housing price and health. Articles were screened by two reviewers for eligibility, which restricted inclusion to original research articles measuring changes in housing prices and health outcomes, published prior to June 31st, 2022.

Among 23 eligible studies, we found that changes in housing prices were heterogeneously associated with physical and mental health outcomes, with multiple mechanisms contributing to both positive and negative health outcomes. Income-level and home-ownership status were identified as key moderators, with lower-income individuals and renters experience negative health consequences from rising housing prices. This may have resulted from increased stress and financial strain among these groups. Meanwhile, the economic benefits of rising housing prices were seen to support health for higher-income individuals and homeowners – potentially due to increased wealth or perception of wealth.

Conclusions

Based on the associations identified in this review, it appears that potential gains to health associated with rising housing prices are inequitably distributed. Housing policies should consider the health inequities born by renters and low-income individuals. Further research should explore mechanisms and interventions to reduce uneven economic impacts on health.

Peer Review reports

Introduction

In contemporary society, the structures we live in, as well as our legal relationships to these structures, are intertwined with our fundamental senses of self and belonging [ 1 , 2 , 3 ]. For decades, homeownership has been recognized as a core measure of success [ 4 , 5 ]. Recognizing the importance of housing, studies have variously examined the effects of wide-ranging housing-related factors on health, including housing quality, overcrowding, neighbourhood deprivation, social cohesion, housing density, housing suitability or sufficiency, and neighbourhood socioeconomic status [ 6 , 7 ]. While these effects continue to be explored, it is generally agreed that housing is a fundamental determinant of health [ 7 ], which broadly exerts impacts on health through a variety of mechanisms.

Indeed, housing-related health effects arise from specific housing conditions, as well as the legal conditions that define our relationships to these spaces, and our emotional attachments to these various factors. For example, living and owning a home can create access to opportunities that can further bolster health [ 8 ]. Similarly, housing related factors—such as indebtedness, mortgage stress, and credit problems—can cause severe mental health problems, depression, and suicide ideation [ 9 , 10 ]. With these factors in mind, people in most countries face numerous barriers to securing their right to a home [ 5 , 11 ], and a wide array of policies have been proposed and implemented to address these barriers [ 12 , 13 , 14 ]. In addition to these factors, the location of a home, the quality of a building, or the neighbourhood context in which a home exists are also hugely influential to health [ 7 , 15 , 16 ].

In conceptualizing these varied mechanisms, it is important to consider both direct and indirect mechanisms through which the relationship between housing and health manifests. Direct effects predominantly emerge from psycho-physiological stress responses. Elevated housing costs can induce chronic stress, leading to mental health conditions, like anxiety and depression, and other health problems [ 17 ]. Indirectly, escalating housing prices exert economic pressures that limit individuals' capacity to allocate resources towards health-promoting activities and necessities. This economic strain can result in compromised nutrition, reduced access to healthcare services, and diminished ability to manage chronic conditions, therefore, exacerbating health disparities. Moreover, the financial burden can lead to other lifestyle changes that further impair physical and mental well-being, such as increased substance use or reduced physical activity.

Despite these effects being documented in previous studies, there are no systematic reviews on the impact of rising housing prices on health. The present review aims to examine the effect of housing price on health by considering whether changes in housing market price impact the health of residents living in an area. To accomplish this aim, we conduct a systematic review. This review is especially timely since housing prices have risen in the past five years at an alarming rate.

Article search

The first step in our multi-stage systematic literature review was to manually identify relevant articles through a rudimentary search on SCOPUS and PubMed ( Appendix B ). We then created a list of keywords to use for our search. Keywords aimed to identify articles that measured changes in housing prices and health impacts, Appendix A outlines how we identified keywords and provides a complete list of selected keywords. After conducting the keyword search in PubMed and SCOPUS, duplicates were removed and the remaining articles were then uploaded to Rayyan, an online software that aids in systematic reviews [ 18 ]. To assess whether our search is comprehensive, AG confirmed that the articles identified in the rudimentary initial search, mentioned earlier, were also included in this search. For the purposes of this literature review, we define health using the language provided by the World Health Organization (1948): “health is a complete state of mental, physical, and social well-being, and not merely the absence of disease.” As such, no additional inclusion or exclusion criteria were used to exclude or include specific health conditions. We felt this was appropriate given that this is the first literature review on this topic and because after a review of included articles, it was apparent that a wide variety of health outcomes have been considered. Furthermore, the biopsychosocial models of health that we engage to inform our view that housing prices have direct and indirect effects on health underscore that diverse and nuanced pathways across various mental and physical domains of health are likely important to consider. Using Rayyan, AG and LW reviewed the titles of each manuscript to remove articles that were clearly not relevant to this review [ 18 ]. The application of inclusion and exclusion criteria resulted in 21 articles that were directly relevant to this review. AG and LW also searched the reference lists for these 21 articles to identify any additional articles. These missed articles were added to our final inclusion list, creating a total of 23 included articles.

Data extraction

Data were extracted by AG and LW from each of the identified and included articles and AG re-reviewed the data extraction to verify accuracy. Extracted variables included: first author name, year of publication, years of data collection, sample size, location(s) of study, study design (e.g., case control, cohort, cross-sectional, serial cross-sectional study), analysis type (e.g., regression), outcome, explanatory factor, confounders/mediators/moderators, and a summary of primary findings (including effect size measures). This data extraction is provided as Table  1 .

Risk of bias assessment

During the data extraction process, we conducted an assessment based on the Joanna Briggs Institute Critical Appraisal Tools [ 42 ]. Each study was classified according to its study design and rated using the appropriate tool designed for each study. However, despite varying methodological quality, no studies were excluded based on risk of bias assessment, as there were no clear sources of systematic bias with sufficient likelihood of challenging the conclusions of the source studies.

Narrative synthesis

During the data extraction and risk of bias assessment phases, AG and LW recorded general notes on each of the studies. These notes, along with the extracted information, were used to construct a narrative synthesis of the evidence. This process was guided by Popay et al.’s [ 43 ] Guidance for Narrative Synthesis in Systematic Reviews. A narrative approach was selected to allow for an examination of the potential complexity inherent in the synthesis of findings across contexts, time periods, and populations to provide a nuanced discussion of what roles housing and rental markets might play in shaping health, with attention to both outcomes and potential mechanisms. Findings within study classes were reviewed to determine potential mediation and moderation. These explorations informed the development of a list of key points used to organize the presentation of our results. We then integrated and contextualized these findings with those from other relevant (though excluded) studies identified through our review process and from the texts of the included articles.

Included studies

Our keyword search returned 6,180 articles. Of these, 5,590 were removed based on review of the abstract and title as they were not directly related to our review topic (i.e., they did not measure changes in housing price and/or health outcomes). The remaining articles were reviewed based on their full-texts and a final list of 26 articles were considered for inclusion. However, five articles were not able to be retrieved (even after emailing the original authors), leaving us with 21 articles. The reference lists and bibliographies for these 21 included articles were then screened and two additional articles were thus included in our review resulting in a final sample of 23 articles. Figure  1 shows the flow diagram for included studies and these studies are listed in Appendix B .

PRISMA systematic review flow diagram

Dates and locations of studies

A full description of studies is included in Table  1 . Studies were published from 2013–2022. Ten studies were from East Asia, eight from the United States, three in Europe, one from Australia, and one included nine countries (France, Japan, Netherlands, Spain, Switzerland, Sweden, United Kingdom, USA).

Study design

Of the included studies, ten had a longitudinal study design, and thirteen studies were serial cross-sectional studies. Studies examined the effect of housing prices on health over time by repeatedly surveying a specific geographical area or population. One study included both qualitative and quantitative data collection.

Outcome variable measurement

Most studies compared multiple outcomes. Seven studies focused on mental health as the outcome variable—utilizing various measures, including self-rated mental health, standardized scales for depression or anxiety, and receipt of pharmaceutical prescriptions [ 22 , 32 , 33 , 35 , 40 , 41 ]. Nine studies analyzed the impact of housing prices on physical health—utilizing various measures of physical health, including objective assessments of physical health (e.g., body mass), self-rated physical health assessments, reports of specific health conditions (e.g., COVID-19), reported health behaviours (e.g., alcohol use, smoking), and mortality ( [ 24 , 28 , 29 , 37 , 44 , 30 , 36 , 38 , 39 ]). Seven studies included both physical and mental health measures as their outcome variable [ 19 , 20 , 23 , 26 , 27 , 34 , 45 ].

Explanatory variable measurement

Housing prices were measured using many different types of data, including house price index, self-reported housing price (extracted from surveys), and average market price. Many studies used house price index as a measure of housing prices [ 19 , 21 , 22 , 30 , 38 , 39 , 41 ]. Zhang & Zhang [ 33 ] included self-reported housing price. Alternatively, many studies examined housing market prices using existing survey data [ 20 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 31 , 32 , 34 , 35 , 36 , 37 , 40 ].

Key findings

Studies included in our review highlighted a plurality of results when testing the relationships between housing prices and health. As shown in Tables  2 (physical health) and Table  3 (mental health), the included articles reported mixed findings across the outcomes explored. Given the heterogeneity of findings regarding the associations between housing prices and health outcomes, several authors examined potential moderators and mediators in attempt to understand the mechanisms at play. These included studies examine the role of wealth effects (by comparing effects on homeowners and renters), socioeconomic status (e.g., income level), and broader economic forces (e.g., area-level improvements). While keeping these pathways in mind, there are likely other alternative explanations beyond those explored. However, these appear to be the most dominant frameworks used to understand the effects in our included studies.

Wealth effects

The first major pathway has been described as a “wealth effect” – which produces different effects for homeowners and renters [ 19 , 20 , 23 , 25 , 27 , 33 , 35 , 37 , 45 ]. For example, Hamoudi & Dowd [ 37 ] report that homeowners living in areas with steep price increases, perceive this as an increase in their overall wealth, resulting in positive health outcomes (not observed for renters). Similarly, Zhang & Zhang [ 33 ] show that increases in house prices has a positive effect on homeowner’s subjective well-being. De & Segura [ 30 ] specifically notes that price depreciation causes homeowners to experience feelings of a loss of wealth, leading to increases in alcohol consumption. Among studies that fail to show a wealth effect, Daysal et al. [ 29 ] shows that rising prices in Denmark do not impact households due to the buffering effects of government supports. Conversely, when examining the effects among renters, Wang & Liang [ 31 ] argue that rising housing prices have detrimental "strain" effect, which is also observed in several studies included in our review [ 25 , 27 , 38 , 39 , 45 ].

Income level

In addition to the wealth and strain effects illustrated through studies among homeowners and renters, many studies also examined the mediating effects of income [ 19 , 20 , 22 , 28 , 29 , 30 , 32 , 33 , 35 , 38 , 39 , 40 ]. Several of these studies show that housing unaffordability constrains spending and that low-income individuals are particularly impacted [ 22 , 24 , 33 , 38 ]. For example, Wong et al. [ 39 ] show that housing prices lead to reduced fruit consumption. However, results also show positive impacts for low-income homeowners – as exemplified by work showing that low-income homeowners are more sensitive to housing price gains [ 38 , 40 ].

Broader economic forces

In considering both mechanisms described above, authors of included studies have also considered whether housing prices are merely an indicator of broader economic trends merits consideration. The most common strategy for accounting for this has been to include other indicators that might capture area level improvements. Indeed, most studies controlled for both individual characteristics or variables, such as age, gender, marital status, years of education, race/ethnicity, and employment status [ 20 , 23 , 24 , 25 , 26 , 27 , 29 , 30 , 32 , 34 , 35 , 37 , 39 , 40 , 41 , 45 ], and a variety of economic factors, including individual income, country-level median income, and local area characteristics [ 19 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 30 , 32 , 36 , 37 , 38 , 39 , 40 , 44 , 45 ]. These factors are important to control for because rising housing prices can indicate a growing economy in which there are substantial improvements to neighbourhoods and communities [ 27 , 29 , 33 , 38 , 40 ]. As such, the observed improvements in health could simply arise from broader economic benefits (rather than being specifically attributable to housing prices) [ 31 , 33 ]. However, generally speaking studies showed that there were independent effects of housing price or value, even after controlling for local area level improvements, and wider economic conditions [ 19 , 27 , 38 ].

Strength of effects

Given heterogeneity in the direction of effects, the lack of standardization in the reporting of effect sizes from study to study, differences in the measurement of exposure and outcome variables, and variation in the inclusion of mediators, moderators, and confounders, we did not conduct a meta-analysis to describe the effect size of housing price on health. However, housing prices appear to exert influence on health and wellbeing with statistically significant effects across various health-related outcomes (See Table  1 for range of effect measures). The effects generally appear to be smaller when considering specific health conditions and greater when considering more subjective and more broad definitions of health (e.g., self-rated health). Of course, at a population-level, even relatively small effect sizes may pose a considerable challenge. For example, Xu & Wang [ 36 ] report that a 10% increase in housing prices is associated with a 6.5% increase in probability of reporting a chronic disease – a relatively small increase on a person-level, but when scaled could easily pose a considerable burden to the health system. In summary, further careful measurement and methodological refinement is needed to quantify the effects of housing prices on various health conditions. For any given health condition, this will require multiple well-designed studies across place and time. Such replication is particularly important given the observed sensitivity of findings to the inclusion of confounders, moderators, and mediators.

Primary findings

While examining whether changes in housing prices are associated with changes in health, we recognized it is difficult to establish a directional and causal relationship between housing price and health. This is particularly true given that health may increase opportunities for home ownership and economic success [ 3 , 46 , 47 , 48 , 49 ]. Nevertheless, given the wealth of literature highlighting housing affordability as a key determinant of health [ 7 , 9 , 10 ], it is reasonably anticipated that rising housing prices would be associated with worse health outcomes among individuals who do not own housing. However, based on analyses of the studies included in this review, the relationship between housing price and health is complex and nuanced, with a significant degree of heterogeneity across outcomes and populations.

First, this review illuminates that changing housing prices impacted different people differently, depending on for example, income level, gender and/or homeownership status [ 19 , 22 , 25 ]. The negative impact of housing price on health for renters and low-income individuals could be due to the existential angst from being excluded from home ownership, which is often considered an important indicator of social class and success [ 8 ]. However, this could also be due to the cost effect that is created from the rising house prices, subsequently raising low-income owners and renters’ cost of living [ 31 ]. Additionally, renting may be associated with lower neighbourhood tenure, especially when individuals are priced out of a neighbourhood [ 8 ]. As a result, they may experience deleterious health effects associated with loneliness, social isolation, lack of neighbourhood cohesion, and community disconnectedness [ 8 ]. Likewise, the positive effect observed among homeowners and high-income individuals may be explained by increases in psychological safety leading to changes in health behaviors, for example, knowing they have invested in a home that will support them or their heirs financially, people may be better able to focus on their well-being. As well, homeowners may be able to directly leverage the value of their home to gain access to additional capital and investment opportunities – which could support increased financial wellbeing [ 8 ].

The effects of housing on health can be conceptualized as arising from two sources. It appears that rising “cost effects” (the increased costs of houses and the costs passed on to tenants) are inversely correlated with health while “wealth effects” (the contributions of housing price to person wealth) contribute positively to health (for example, for homeowners and investors whose wealth increases due to the rising cost of housing). The balance of these effects differs depending on their unique impact on individuals – with lower income people and renters more strongly impacted by cost effects, and higher income people and homeowners more strongly impacted by wealth effects.

In considering these effects, we note that there is likely considerable geographic, temporal, and contextual variation in the health effects of rising housing prices. For example, rising housing prices may occur alongside neighbourhood improvements (or degradation) and economic booms (or recessions) [ 45 ], which themselves are associated with improvements (or deterioration) to health [ 8 ]. As such, the presence of these factors may obscure or interact with the gains to health. Similarly, variations across cultures and countries may change how individuals internalize the rising housing prices [ 50 ], causing them to experience greater or lesser distress in reaction to rising prices.

Limitations of included studies and directions for future research

Given these two primary factors, research highlights several opportunities for improving this literature. First, future studies should give more careful attention to how moderators and mediators are conceptualized. For example, “home-owners” are hardly a homogenous class of individuals: some own their homes outright and others are paying mortgages that offer varying levels of security (e.g., fixed vs. variable mortgages, 5-year vs. 30-year mortgages) [ 8 ]. Second, a broader range of effect moderators should be explored. For example, few studies specifically examined the health effects of rising home prices on vulnerable populations, including young adults and first time home buyers who may be especially disadvantaged by rapidly growing housing prices [ 50 ]. Similarly, isolating effects as arising from economic, legal, environmental, and social pathways can help identify strategies for mitigating health harms. For example, it may be important to understand whether changes to neighbourhood environments drive health harms as opposed to changes in personal financial status. Third, more within-person studies are needed to understand the potential mechanisms and situational factors that promote or mitigate the health effects of rising housing prices. Along with use of appropriate, theoretically informed moderators, isolating the within-person effects can help us better quantify the effects of interest to inform policies and prevention strategies. Fourth, longer follow-up times may allow for better understanding regarding the time-horizons of the effects explored. Indeed, it is possible that rising housing prices could have differential effects on the health of a population in the short versus long term. This is particularly important given the interaction between housing prices (which may act as a price signal for investments) and other economic factors with strong potential to increase health [ 8 ]. Fifth, the studies used a variety of measures for housing price and health outcomes – which varied in quality. For example, health outcomes were primarily measured using self-reported measures [ 19 , 20 , 23 , 25 , 26 , 27 , 30 , 31 , 33 , 34 , 35 , 37 , 38 , 39 , 40 , 41 ] – which may be highly sensitive to bias due to the likelihood that individuals might report worse health when they are unhappy with economic factors. Improving measurement of outcomes can be done by leveraging administrative and other data sources. Sixth, it can be difficult to link area-level and individual-level factors, particularly in the context of limited cross-sectional studies or in longitudinal studies with only a few follow-up points. Likewise, many cohort-based studies have limited geographic coverage or insufficient temporal scope. As such, longer, larger, and wider studies are needed to fully ascertain the relationships under consideration.

Implications of findings

Although further research is required to overcome the limitations mentioned, existing evidence indicates that increases in housing prices may significantly influence health outcomes. Future studies should aim to exclude alternative explanations, examine the effects over longer periods, and establish consistent measurement methods to predict the impact of housing prices more accurately on health. The findings of those students will aid policymakers in creating strategies that address the health implications of rising housing prices. Policy makers should develop frameworks that respond to the impacts of rising housing prices on health. Such approaches could be facilitated through frameworks such as the WHO’s Health in All Policies (HiAP) policy, which advocates for the inclusion of health and social impacts among other criteria used throughout decision making processes [ 51 ]. Many studies in this review support this view and describe their work as having important implications for housing and health policy [ 19 , 20 , 21 , 24 , 26 , 27 , 28 , 29 , 31 , 34 , 35 , 37 , 38 , 39 ]. For example, Yuan et al. [ 26 ] notes the importance of directing government support and housing subsidies towards vulnerable groups – though these should be packaged with other policies [ 26 ]. Such supports can apparently buffer against the negative effects of rising housing prices by creating a saftey net that reduces the psychosocial and cognitive effects associated with economic changes in one's personal circumnstance. Arcaya et al. [ 21 ] also recommends governments investigate establishing more mental health facilities in areas where housing price fluctuations impact people's mental health but warns economic development that allows for greater investment in health infrastructure can also lead to increases in housing prices.

Of course, other types of interventions may also be warranted, including broader financial interventions (e.g., direct loans; [ 52 ], those which promote community, neighborhood, and social cohesion among residents [ 53 ], or those that aim to change how people value home ownership [ 26 ]. With respect to this final option, communities should consider whether renting may in fact be a desirable outcome for some individuals and therefore promote a culture in which individuals realize the variety of investment opportunities available to them rather than being overly-focused on a traditional model of investment [ 26 , 32 ]. For example, Zhang & Zhang [ 33 ] writes that homeowners should be provided financial and economic knowledge to better manage wealth gains, however, this could be taken one step further and include the importance of educating people on the dangers on the commodification of housing, to prevent an over reliance on the importance of housing wealth gains.

Limitations of our review

In addition to the limitations specific to studies included in our review, our review itself also has several limitations. First, while we trained two reviewers to conduct article screening, assessed inter-rater reliability as greater than 80%, and adjudicated conflicts with the help of a third reviewer, it is possible some articles that could have been included were excluded due to the many different forms of outcome and explanatory variable measurement. Second, while we have searched multiple databases, used comprehensive key words for our search, and conducted manual searches of the reference lists, it is possible there were studies that we missed and failed to include in this study. That said, it is unlikely the exclusion of these articles would change our conclusion that the literature is currently mixed and that there is a need to flesh out the mechanisms and moderators that link housing prices to health. Third, we were not able to conduct a meta-analysis, and our numeric reports of the number of studies with each characteristic should not be treated as a meta-analysis. Rather, these findings and analyses of these studies should be interpreted as a descriptive analysis that highlights significant heterogeneity of findings and critical inconsistencies in the mechanisms, mediators, and moderators that give rise to these associations. Fourth, we did not exclude any studies based on article quality because we did not find there to be sufficient heterogeneity in the quality of the observational studies to merit exclusion according to variable inclusion, study design, or sampling method. In other words, we sought to avoid introducing bias by arbitrarily excluding articles – particularly given that the number of articles captured here was already relatively low (at least given the diversity of methods, measures, and populations captured). However, future reviews might consider more narrowed inclusion and exclusion criteria when a sufficient body of literature is available for a given outcome. For example, limiting analyses to only well-designed cohort studies might support a more careful selection of articles. Finally, we note that we included studies across a wide variety of health outcomes. While this was done to maximize inclusion (given the wide heterogeneity of measures used), we acknowledge that future research might be strengthened by studying specific pathways linking housing prices to specific health and social outcomes. Such detailed research is greatly needed so as to not only highlight the relevance of housing prices to health but identify strategies for mitigating potential harms of rising housing prices.

Our review shows that there are complex relationships between housing prices and health – with studies arriving to mixed conclusions across a wide-variety of health outcomes and populations. Yet, there is insufficient evidence for a causal relationship, but it appears that if such a relationship exists it likely differs according to homeownership status, income-level, and as a factor of the broader economic and structural forces in play, including the level of economic supports provided by governments for low income individuals. Future research should explore these pathways, moderators, and confounders using long-term, geographically diverse, cohort studies that account for a broad diversity of causal or alternative mechanisms. Such future research will allow for a more nuanced understanding of health and health inequities related to rising housing prices.

Availability of data and materials

All data generated or analysed during this study are included in this published article and its supplementary information files.

Saunders P. The meaning of ‘home’ in contemporary english culture. Hous Stud. 1989;4:177–92.

Article   Google Scholar  

Shields M. Community belonging and self-perceived health. Health Rep. 2008;19:51–60.

PubMed   Google Scholar  

Smith A. Exploring the interrelationship between the meanings of homeownership and identity management in a liquid society: a case study approach. [Doctoral dissertation, Keele University] Keele Business School. 2017.

Després C. The meaning of home: literature review and directions for future research and theoretical development. Locke Sci Publ Co Inc. 1991;8:96–115.

Goodman LS, Mayer C. Homeownership and the American Dream. J Econ Perspect. 2018;32:31–58.

Kawachi I. Social Capital and Community Effects on Population and Individual Health. Ann N Y Acad Sci. 1999;896:120–30.

Article   CAS   PubMed   Google Scholar  

Rolfe S, Garnham L, Godwin J, Anderson I, Seaman P, Donaldson C. Housing as a social determinant of health and wellbeing: developing an empirically-informed realist theoretical framework. BMC Public Health. 2020;20:1138.

Article   PubMed   PubMed Central   Google Scholar  

Rohe WM, Van Zandt S, McCarthy G. Home Ownership and Access to Opportunity. Hous Stud. 2002;17:51–61.

Turunen E, Hiilamo H. Health effects of indebtedness: a systematic review. BMC Public Health. 2014;14:489.

Waldron R, Redmond D. “We’re just existing, not living!” Mortgage stress and the concealed costs of coping with crisis. Hous Stud. 2017;32:584–612.

Lemphers B. Barriers & Opportunities; A Meta-Analysis of Affordable Rental Housing In Canada. Dalhousie University; 2017.  https://cdn.dal.ca/content/dam/dalhousie/pdf/faculty/architecture-planning/school-of-planning/pdfs/studentwork/MPlanIndependentStudy/Lemphers-Barriers%20and%20Opportunity-A%20Meta-Analysis%20of%20Affordable%20Rental%20Housing%20in%20Canada-2017.pdf .

Canadian Mortgage and Housing Association. Demand or Supply Side Housing Assistance?. 2019. https://www.cmhc-schl.gc.ca/professionals/housing-markets-data-and-research/housing-research/research-reports/accelerate-supply/research-insight-updating-debate-between-demand-supply-side-housing-assistance .

Glaeser EL. Rethinking the Federal Bias Toward Homeownership. SSRN Electron J. 2011. https://doi.org/10.2139/ssrn.1914468 .

Pomeroy S. Optimizing Demand and Supply Side Housing Assistance Programs. 2016. Carleton University Centre for Urban Research and Education (CURE). https://www.focus-consult.com/wp-content/uploads/Policy-Brief_Optimizing-Supply-and-Demand-Measures-2016.pdf .

Ige-Elegbede J, Pilkington P, Orme J, Williams B, Prestwood E, Black D, et al. Designing healthier neighbourhoods: a systematic review of the impact of the neighbourhood design on health and wellbeing. Cities Health. 2022;6:1004–19.

Article   PubMed   Google Scholar  

Pérez E, Braën C, Boyer G, Mercille G, Rehany É, Deslauriers V, et al. Neighbourhood community life and health: A systematic review of reviews. Health Place. 2020;61: 102238.

Çalıyurt O. The Mental Health Consequences of the Global Housing Crisis. Alpha Psychiatry. 2022;23:264–5.

Rayyan. 2022. https://www.rayyan.ai

Yue D, Ponce NA. Booms and Busts in Housing Market and Health Outcomes for Older Americans. Innov Aging. 2021;5:igab012.

Chen N, Shen Y, Liang H, Guo R. Housing and Adult Health: Evidence from Chinese General Social Survey (CGSS). Int J Environ Res Public Health. 2021;18:916.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Arcaya MC, Nidam Y, Binet A, Gibson R, Gavin V. Rising home values and Covid-19 case rates in Massachusetts. Soc Sci Med. 2020;265: 113290.

Lee CY, Chen PH, Lin YK. An Exploratory Study of the Association between Housing Price Trends and Antidepressant Use in Taiwan: A 10-Year Population-Based Study. Int J Environ Res Public Health. 2021;18:4839.

Fichera E, Gathergood J. Do Wealth Shocks Affect Health? New Evidence from the Housing Boom. Health Econ. 2016;25:57–69.

Kim I. Spatial distribution of neighborhood-level housing prices and its association with all-cause mortality in Seoul, Korea (2013–2018): A spatial panel data analysis. SSM - Popul Health. 2021;16: 100963.

Hamoudi A, Dowd JB. Housing Wealth, Psychological Well-being, and Cognitive Functioning of Older Americans. J Gerontol Ser B. 2014;69:253–62.

Yuan W, Gong S, Han Y. How does rising housing price affect the health of middle-aged and elderly people? The complementary mediators of social status seeking and competitive saving motive. PLoS ONE. 2020;15: e0243982.

Atalay K, Edwards R, Liu BYJ. Effects of house prices on health: New evidence from Australia. Soc Sci Med. 2017;192:36–48.

Bao W, Tao R, Afzal A, Dördüncü H. Real Estate Prices, Inflation, and Health Outcomes: Evidence From Developed Economies. Front Public Health. 2022;10: 851388.

Daysal NM, Lovenheim M, Siersbæk N, Wasser DN. Home prices, fertility, and early-life health outcomes. J Public Econ. 2021;198:104366.

De PK, Segura-Escano R. Drinking during downturn: New evidence from the housing market fluctuations in the United States during the Great Recession. Econ Hum Biol. 2021;43: 101070.

Wang H-Q, Liang L-Q. How Do Housing Prices Affect Residents’ Health? New Evidence From China. Front Public Health. 2022;9: 816372.

Wei G, Zhu H, Han S, Chen J, Shi L. Impact of house price growth on mental health: Evidence from China. SSM - Popul Health. 2021;13: 100696.

Zhang C, Zhang F. Effects of housing wealth on subjective well-being in urban China. J Hous Built Environ. 2019;34:965–85.

Feng Y, Nie C. The effect of housing price on health: new evidence from China. Appl Econ Lett. 2022;29:1439–46.

Chun H. Do housing price changes affect mental health in South Korea? Ethiop J Health Dev. 2020;34:48–59.

Google Scholar  

Xu Y, Wang F. The health consequence of rising housing prices in China. J Econ Behav Organ. 2022;200:114–37.

Hamoudi A, Dowd JB. Physical health effects of the housing boom: quasi-experimental evidence from the health and retirement study. Am J Public Health. 2013;103:1039–45.

Sung J, Qiu Q. The Impact of Housing Prices on Health in the United States Before, During, and After the Great Recession. South Econ J. 2020;86:910–40.

Wong ES, Oddo VM, Jones-Smith JC. Are Housing Prices Associated with Food Consumption? Int J Environ Res Public Health. 2020;17:3882.

Ratcliffe A. Wealth Effects, Local Area Attributes, and Economic Prospects: On the Relationship between House Prices and Mental Wellbeing. Rev Income Wealth. 2015;61:75–92.

Joshi NK. Local house prices and mental health. Int J Health Econ Manag. 2016;16:89–102.

Joanna Briggs Institute. Critical Appraisal Tools. 2022. https://jbi.global/critical-appraisal-tools

Popay J, Roberts H, Sowden AJ, Petticrew M, Arai L, Rodgers ME., Britton, N. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews. 2006. https://www.lancaster.ac.uk/media/lancaster-university/content-assets/documents/fhm/dhr/chir/NSsynthesisguidanceVersion1-April2006.pdf .

Arcaya MC, Nidam Y, Binet A, Gibson R, Gavin V. Rising home values and Covid-19 case rates in Massachusetts. Soc Sci Med. 1982;2020(265): 113290.

Wang H-Q, Liang L-Q. How Do Housing Prices Affect Residents’ Health? New Evidence From China. Front Public Health. 2022;9:1–11.

Shlay AB. Low-income Homeownership: American Dream or Delusion? Urban Stud. 2006;43:511–31.

Boehm TP, Schlottmann AM. The dynamics of race, income, and homeownership. J Urban Econ. 2004;55:113–30.

Fazel S, Khosla V, Doll H, Geddes J. The Prevalence of Mental Disorders among the Homeless in Western Countries: Systematic Review and Meta-Regression Analysis. PLOS Med. 2008;5: e225.

McConnell ED. Who has Housing Affordability Problems? Disparities in Housing Cost Burden by Race, Nativity, and Legal Status in Los Angeles. Race Soc Probl. 2013;5:173–90.

Ding Y, Chin L, Lu M, Deng P. Do high housing prices crowd out young professionals?—Micro-evidence from China. Econ Res-Ekon Istraživanja. 2022;36(2):1–19.

CAS   Google Scholar  

World Health Organization. 2023. https://www.who.int/about/accountability/governance/constitution .

Fernald LC, Hamad R, Karlan D, Ozer EJ, Zinman J. Small individual loans and mental health: a randomized controlled trial among South African adults. BMC Public Health. 2008;8:409.

Silverstein M, Cong Z, Li S. Intergenerational Transfers and Living Arrangements of Older People in Rural China: Consequences for Psychological Well-Being. J Gerontol Ser B. 2006;61:S256–66.

Download references

Acknowledgements

We would like to acknowledge the support of Logan White for his support in conducting the literature review.

KGC is supported by a Michael Smith Health Research BC Scholar Award. This project was funded by grants from The Canadian Institutes for Health Research and the GenWell Project.

Author information

Authors and affiliations.

Faculty of Health Sciences, Simon Fraser University, Blusson Hall, 8888 University Dr. , Burnaby, BC, V5A 1S6, Canada

Ashmita Grewal, Kirk J. Hepburn, Scott A. Lear & Kiffer G. Card

Vancouver School of Economics, University of British Columbia, Vancouver, BC, Canada

Marina Adshade

You can also search for this author in PubMed   Google Scholar

Contributions

KGC & AG conceptualized the study design. KGC, LW, and AG conducted the literature review, search, and data extraction. KGC & AG drafted the initial manuscript. All authors provided substantive intellectual and editorial revisions and approved the final manuscript.

Corresponding author

Correspondence to Ashmita Grewal .

Ethics declarations

Ethics approval and consent to participate.

This study does not require ethics approval as it is a literature review.

Consent for publication

There are no figures or materials presented in this manuscript that require consent.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Grewal, A., Hepburn, K.J., Lear, S.A. et al. The impact of housing prices on residents’ health: a systematic review. BMC Public Health 24 , 931 (2024). https://doi.org/10.1186/s12889-024-18360-w

Download citation

Received : 04 August 2023

Accepted : 14 March 2024

Published : 01 April 2024

DOI : https://doi.org/10.1186/s12889-024-18360-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Housing price
• Economic stress
• Social determinants of health

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.2(6); 2022 Nov

Effective Peer Review: Who, Where, or What?

Peer review is widely viewed as one of the most critical elements in assuring the integrity of scientific literature ( Baldwin, 2018 ; Smith, 2006 ). Despite the widespread acceptance and utilization of peer review, many difficulties with the process have been identified ( Hames, 2014 ; Horrobin, 2001 ; Smith, 2006 ). One of the primary goals of the peer review process is to identify flaws in the work and, by so doing, help editors choose which manuscripts to publish. It is surprising that one of the persistent problems in peer review is assessing the quality of the reviews. Both authors and journal editors expect peer review to detect errors in experimental design and methodology and to ensure that the interpretation of the findings is presented in an objective and thoughtful manner. In traditional peer review, two or more reviewers are asked to evaluate a manuscript on the basis of the expectation that if the two reviewers agree on the quality of the submission, the likelihood of a high-quality review is increased. Unfortunately, studies have not consistently confirmed a high degree of agreement among reviewers. Rothwell and Martynn (2000) evaluated the reproducibility of peer review in neuroscience journals and meeting abstracts and found that agreement was approximately what would be expected by chance. Similarly, Scharschmidt et al. (1994) found similar results in the evaluation of 1,000 manuscripts submitted to the Journal of Clinical Investigation, where clustering of grades in the middle resulted in an agreement being “…only marginally…” better than chance. These observations suggest that we cannot rely on the agreement of reviewers to be an indication of the quality of the reviews. Another potential way to evaluate the quality of reviews would be to assess the ability of reviewers to detect errors in submissions. It is generally accepted that detection of intentional fraud is beyond the scope of typical peer review, but we do expect reviewers to detect major and minor errors as a primary function of the traditional peer review system ( Hwang, 2006 ; Weissman, 2006 ). Schroter et al. (2008) evaluated the ability of reviewers to detect major and minor errors by introducing errors into three previously published papers describing randomized controlled clinical trials. Reviewers detected approximately three of the nine errors introduced in each manuscript. Unfortunately, reviewers who had undergone training in how to conduct a high-quality peer review were not significantly better than untrained reviewers. Similar results have been reported by Godlee et al. (1998) and Baxt et al. (1998) . Baxt et al. (1998) did report that reviewers who rejected or suggested revision of a manuscript identified more errors than those who accepted the manuscript (decision: 17.3% of major errors detected [accept], 29.6% of major errors detected [revise], and 39.1% of major errors detected [reject]). It is almost certainly true that the extent of the failure to recognize errors in submitted manuscripts may differ among scientific disciplines and journals. It also however seems likely that these observations do have some applicability to journals such as JID Innovations . It is critical that both authors and editors are cognizant of these limitations of peer review in their assessment of reviews. These findings compel journals to continue to work to develop new strategies to train and evaluate reviewers. The findings also suggest that factors beyond the failure to detect objective mistakes in a manuscript may be playing a role in the discrepancy in reviewers’ evaluations. One area of ongoing concern in the peer review process is the role of reviewer bias in assessing the scientific work of colleagues ( Kuehn, 2017 ; Lee et al, 2013 ; Tvina et al, 2019 ).

Bias in the peer review process can take many forms, including collaborator/competitor bias, affiliation bias based on an investigator’s institution or department, geographical bias based on the region or country of origin, racial bias, and gender or sex bias ( Kuehn, 2017 ; Lee et al, 2013 ; Tvina et al, 2019 ). All of these forms of bias present the risk that a decision of the reviewer will not be based solely on the quality or merit of the work but rather be influenced by a bias of the reviewer. We and other journals routinely seek to avoid selecting individuals to review work from their own institutions and ask all reviewers to declare any potential personal conflicts of interest. All these methods require either the editor or the reviewer to identify a bias and fail to address the issue of implicit or unconscious reviewer bias. The dominant method currently utilized for peer review is the so-called single-blind review, in which the identity and affiliations of the authors are known to the reviewers, whereas the identity of reviewers remains unknown to the authors. This has led to concern that knowledge of the identity of the authors and their institutions may be the source of significant reviewer bias, especially implicit bias, in the evaluation of manuscripts. Double anonymized peer review (DAPR), also known as double-blind peer review, has been suggested as a way to address this issue ( Bazi, 2020 ; Lee et al, 2013 ). Studies have compared single-blind with double-blind reviewing and reported that there is no significant difference in the quality of the reviews ( Alam et al, 2011 ; Godlee et al, 1998 ; Justice et al, 1998 ; van Rooyen et al, 1998 ). Although these studies looked at measures such as the number of errors detected, acceptance rate, and distribution of initial reviewer scores, they were not designed to address specific sources of bias such as authors’ gender, institution, or geographic location. Other studies have been undertaken to directly address the issue of bias in the peer review process. Ross et al (2006) compared the acceptance of abstracts submitted to the American Heart Association’s annual scientific meeting during a period when the reviewers knew the identity and origin of the authors (i.e., single-blind review) with when this information was not known by the reviewers (i.e., double-anonymized peer review). They found a significant increase in acceptance of non‒United States abstracts and abstracts from non-English speaking countries when the reviewers were unaware of the country of origin of the abstracts ( Ross et al, 2006 ). They also found a significant decrease in the acceptance of abstracts from prestigious institutions when the reviewers were unaware of the institutions where the work was done. In a similar study, Tomkins et al. (2017) found that papers submitted to a prestigious computer science meeting were more likely to be accepted if they were from famous authors, top universities, and top companies. Okike et al. (2016) documented similar results for manuscripts submitted to the orthopedic literature. They submitted a fabricated manuscript that was presented as being written by two prominent orthopedic surgeons (past Presidents of the American Academy of Orthopedic Surgeons) from prestigious institutions. When reviewed in the traditional single-blind fashion, which included the identity of the authors, the manuscript was accepted by 87% of the reviewers. By contrast, when the identity of the authors was unknown, the manuscript was accepted by 68% of the reviewers ( P  = 0.02) ( Blank, 1991 ). A study conducted at The American Economic Review found that authors at near-top-ranked universities experienced lower acceptance rates when authorship was anonymized ( Blank, 1991 ). Of interest, they also found that for women, there was no difference in the acceptance rate between the double-anonymized and single-blinded reviews; however, for men, the acceptance rate was lower with double-anonymized reviews.

These studies provide strong evidence that knowledge of who and where the study was performed can impact the acceptance of abstracts and manuscripts. This conflicts with the goal of the review process to base our judgments on the quality of what the results demonstrate. It is difficult to estimate how much this may affect the fate of a manuscript at JID Innovations . We do not have evidence that our review process has been impacted by bias as is reported in the studies discussed. However, neither can we state with certainty that such bias is not a factor in the reviews we receive. One of the goals of JID Innovations is to be a truly open-access journal available to all investigators in skin science from around the world. We have sought to be an outlet for studies that challenge existing paradigms or that may report negative results. We want to be seen as providing fair and objective reviews for all authors, regardless of where they work or who they are. If we are to achieve this goal, it is imperative that the who and where of a specific manuscript do not negatively impact the evaluation of the what. We want young investigators, investigators at less prestigious institutions or from less well-known laboratories, and investigators from any country around the world to be confident that their work will be judged by what they report and not by the who and the where.

To be true to this mission, JID Innovations will be initiating DAPR starting in October 2022. This is not being done because we are aware of any issues of bias with our current process of peer review but because we realize that the absence of proof is not proof of absence. As a part of this process, authors will be asked to remove identifying material from manuscripts at the time of submission in preparation for the review process ( https://www.jidinnovations.org/content/authorinfo ). As a result, primary reviewers will see only the what of the manuscript. We realize that this process involves extra work for both the authors and our staff, but we feel the benefits will outweigh this small cost. Indeed, in other journals that have taken this step, surveys have shown that both authors and reviewers ultimately prefer double-anonymized reviews ( Bennett et al, 2018 ; Moylan et al, 2014 ). We realize that achieving 100% anonymization of a manuscript is nearly impossible. Studies have shown that the rate of successful anonymizing, where the reviewers cannot discern the authorship of a manuscript, ranged from 47 to 73%. It is however interesting that even with this rate of success in the anonymizing process, a meta-analysis of trials of double- versus that of single-blind peer review has suggested an impact, with lower acceptance rates with double-anonymized peer review ( Ucci et al, 2022 ). More work clearly needs to be done to assess the value of the DAPR process, and we will be monitoring our results carefully.

The institution of DAPR in JID Innovations will assure our authors that the what of their manuscript is our focus. It does not matter who you are or where you are from. It will also emphasize to our reviewers that our focus is on the what. We will be carefully monitoring the results of this new policy and plan to report back on our experience. We also welcome your feedback on your experience as a reviewer and author for JID Innovations ; send your comments to us at [email protected] .

Finally, this decision should be seen not as the end of our efforts to improve the peer review process but merely as a first step. We will continue to work to improve all aspects of the peer review process for JID Innovations . We firmly believe that the use of double-blind -anonymized peer review will bring us closer to ensuring to our authors and readers that the work that is published by JID Innovations has been selected on the basis of what the paper reports and not on who performed the studies or where they were located.

Conflict of Interest

The author states no conflicts of interest.

• Alam M., Kim N.A., Havey J., Rademaker A., Ratner D., Tregre B., et al. Blinded vs. unblinded peer review of manuscripts submitted to a dermatology journal: a randomized multi-rater study. Br J Dermatol. 2011; 165 563‒7. [ PubMed ] [ Google Scholar ]
• Baldwin M. Scientific autonomy, public accountability, and the rise of “peer review” in the Cold War United States. Isis. 2018; 109 :538–558. [ Google Scholar ]
• Baxt W.G., Waeckerle J.F., Berlin J.A., Callaham M.L. Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Ann Emerg Med. 1998; 32 310‒7. [ PubMed ] [ Google Scholar ]
• Bazi T. Peer review: single-blind, double-blind, or all the way-blind? Int Urogynecol J. 2020; 31 :481–483. [ PubMed ] [ Google Scholar ]
• Bennett K.E., Jagsi R., Zietman A. Radiation oncology authors and reviewers prefer double-blind peer review. Proc Natl Acad Sci USA. 2018; 115 :E1940. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Blank R.M. The effects of double-blind versus single-blind reviewing: experimental evidence from the American Economic Review. Am Econ Rev. 1991; 81 :1041–1067. [ Google Scholar ]
• Godlee F., Gale C.R., Martyn C.N. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA. 1998 280237‒40. [ PubMed ] [ Google Scholar ]
• Hames I. Peer review at the beginning of the 21st century. Sci Ed. 2014; 1 :4–8. [ Google Scholar ]
• Horrobin D.F. Something rotten at the core of science? Trends Pharmacol Sci. 2001; 22 51‒2. [ PubMed ] [ Google Scholar ]
• Hwang W.S. Can peer review police fraud? Nat Neurosci. 2006; 9 :149. [ PubMed ] [ Google Scholar ]
• Justice A.C., Cho M.K., Winker M.A., Berlin J.A., Rennie D. Does masking author identity improve peer review quality? A randomized controlled trial. PEER Investigators [published correction appears in JAMA 1998;280:968. JAMA. 1998; 280 240‒2. [ PubMed ] [ Google Scholar ]
• Kuehn B.M. Rooting out bias. ELife. 2017; 6 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lee C.J., Sugimoto C.R., Zhang G., Cronin B. Bias in peer review. JASIST. 2013; 64 :2–17. [ Google Scholar ]
• Moylan E.C., Harold S., O'Neill C., Kowalczuk M.K. Open, single-blind, double-blind: which peer review process do you prefer? BMC Pharmacol Toxicol. 2014; 15 :55. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Okike K., Hug K.T., Kocher M.S., Leopold S.S. Single-blind vs double-blind peer review in the setting of author prestige. JAMA. 2016; 316 1315‒6. [ PubMed ] [ Google Scholar ]
• Ross J.S., Gross C.P., Desai M.M., Hong Y., Grant A.O., Daniels S.R., et al. Effect of blinded peer review on abstract acceptance. JAMA. 2006; 295 1675‒80. [ PubMed ] [ Google Scholar ]
• Rothwell P.M., Martyn C.N. Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? Brain. 2000; 123 1964‒9. [ PubMed ] [ Google Scholar ]
• Scharschmidt B.F., DeAmicis A., Bacchetti P., Held M.J. Chance, concurrence, and clustering. Analysis of reviewers' recommendations on 1,000 submissions to the Journal of Clinical Investigation. J Clin Invest. 1994; 93 1877‒80. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Schroter S., Black N., Evans S., Godlee F., Osorio L., Smith R. What errors do peer reviewers detect, and does training improve their ability to detect them? J R Soc Med. 2008; 101 507‒14. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Smith R. Peer review: a flawed process at the heart of science and journals. J R Soc Med. 2006; 99 178‒82. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Tomkins A., Zhang M., Heavlin W.D. Reviewer bias in single- versus double-blind peer review. Proc Natl Acad Sci USA. 2017; 114 12708‒13. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Tvina A., Spellecy R., Palatnik A. Bias in the peer review process: can we do better? Obstet Gynecol. 2019; 133 1081‒3. [ PubMed ] [ Google Scholar ]
• Ucci M.A., D'Antonio F., Berghella V. Double- vs single-blind peer review effect on acceptance rates: a systematic review and meta-analysis of randomized trials. Am J Obstet Gynecol MFM. 2022; 4 [ PubMed ] [ Google Scholar ]
• van Rooyen S., Godlee F., Evans S., Smith R., Black N. Effect of blinding and unmasking on the quality of peer review: a randomized trial. JAMA. 1998; 280 234‒7. [ PubMed ] [ Google Scholar ]
• Weissmann G. Science fraud: from patchwork mouse to patchwork data. FASEB J. 2006; 20 587‒90. [ PubMed ] [ Google Scholar ]

• Open access
• Published: 05 April 2024

Examining provider practice-level disparities in delivery outcomes among patients with a history of Cesarean Delivery

• Lily McCarthy 1 ,
• Nicola F Tavella 2 , 4 ,
• Sara Wetzler 1 ,
• Lily Ardente 1 ,
• Molly Chadwick 3 ,
• Dexter Paul 2 ,
• Nikki Sabet 2 ,
• Toni Stern 2 &
• Angela Bianco 2 , 4  

BMC Pregnancy and Childbirth volume  24 , Article number:  243 ( 2024 ) Cite this article

126 Accesses

5 Altmetric

Metrics details

Choosing whether to pursue a trial of labor after cesarean (TOLAC) or scheduled repeat cesarean delivery (SRCD) requires prenatal assessment of risks and benefits. Providers and patients play a central role in this process. However, the influence of provider-associated characteristics on delivery methods remains unclear. We hypothesized that different provider practice groups have different obstetric outcomes in patients with one prior cesarean delivery (CD).

This was a retrospective cohort study of deliveries between April 29, 2015 – April 29, 2020. Subjects were divided into three cohorts: SRCD, successful VBAC, and unsuccessful VBAC (patients who chose TOLAC but had a CD). Disparities were reviewed between five different obstetric provider practice groups, determined from a breakdown of different providers delivering at the study site during the study period. Proportional differences were examined using Chi-squared tests and logistic regression models.

1,439 deliveries were included in the study. There were significant proportional disparities between patients in the different groups. Specifically, patients from Group D were significantly more likely to undergo successful VBAC, while patients seeing a provider from Group A were more likely to deliver by SRCD. In our multivariate analysis of successful versus unsuccessful VBAC, patients from Group D had greater odds ratios of successful VBAC compared to Group A. Patients delivered by Group E had a significantly lower odds ratio of successful VBAC.

This study suggests an association between provider practice groups and delivery outcomes among patients with one prior CD. These data contribute to a growing body of literature around patient choice in pregnancy and the interplay of patients and providers. These findings help to guide future investigations to improve outcomes among patients with a history of CD.

Peer Review reports

Cesarean deliveries (CDs), including scheduled repeat cesarean deliveries (SRCDs), have become more common [ 1 , 2 ]. However, a trial of labor after cesarean (TOLAC) is safe for most women [ 3 ]. Successful vaginal birth after cesarean (VBAC) is associated with lower maternal mortality and morbidity, faster recovery, and decreased future complications [ 4 , 5 , 6 ]. Nevertheless, morbidity outcomes from unsuccessful VBAC may be worse than those of a SRCD, particularly in the case of uterine rupture [ 7 ]. One recent study found uterine rupture during TOLAC associated with a host of adverse maternal and neonatal health outcomes [ 8 ]. Thus, the choice of delivery method involves risk-benefit analyses during shared decision-making discussions between patients and providers.

There is emerging interest in how provider differences may influence decisions about delivery mode [ 9 , 10 , 11 ]. Recent research reaffirms that patients often defer to their providers regarding delivery [ 12 ]. TOLAC preference appears more common among midwives, while SRCD is more prevalent among obstetricians (OBs) [ 13 , 14 , 15 , 16 ]. TOLAC also seems more prevalent among laborists (attending physicians employed by the institution to staff the labor floor and tend to any patient in labor) [ 17 ]. Other factors related to provider characteristics including differences in call schedule and affective traits have also been analyzed [ 10 , 18 ]. Differences in preferences between providers and patients have been reported, with qualitative influences on delivery [ 19 ].

Nevertheless, there are still lingering questions about provider practice-related dynamics. Little is known about the potential effects of differences in patient preferences and provider practice settings (including private general practices and maternal-fetal medicine practices). One study evaluated the significant implications of patient cultural factors on delivery mode preference [ 20 ]. Another study examined factors associated with patient demand for CD, with a history of infertility increasing likelihood of CD [ 21 ]. In the current study, our primary goal was to determine whether differences in delivering provider practice correlated with higher likelihood of TOLAC or SRCD. We hypothesized that patients with a history of cesarean delivered by different provider practice groups would experience different rates of outcomes related to mode of delivery and associated demographic and clinical characteristics.

This was a retrospective cohort study of patients with a history of CD who delivered at a single major urban hospital in New York City between April 29, 2015 – April 29, 2020. For reference, in 2022 this institution conducted 6,956 deliveries, of which 27.8% were by CD. We serve a large, diverse population of patients with varying levels of social vulnerability. The Icahn School of Medicine at the Mount Sinai Hospital Institutional Review Board (IRB) oversaw the conduct of this project. Chart review abstracted demographic and clinical data, including race and ethnicity, health insurance, pre-pregnancy body mass index (BMI), third-trimester sonographic estimated fetal weight (EFW), census-tract specific Social Vulnerability Index (SVI), and provider-level information related to the delivering clinician. SVI values were dichotomized as “low-medium” if the index value was < 0.75 and “high” if the value was \(\ge\) 0.75. We excluded patients with multiple gestations and a history of more than one CD, as well as patients who delivered by cesarean for contraindications of TOLAC (malpresentation, placenta previa, vasa previa, history of myomectomy, suspected placenta accreta spectrum, fetal anomalies, and unplanned clinical indications including hypertensive disorders, cardiac abnormalities, and active bleeding). Patients were grouped based on whether they opted for TOLAC and delivered by VBAC or CD. This yielded three groups: successful VBAC, unsuccessful VBAC, and SRCD. We divided the study sample in this way to examine odds of successful versus unsuccessful VBAC among different provider practice groups.

The five categories of delivering provider practices included Group A, private general obstetrician groups of varying sizes; Group B, a large non-profit OB practice serving a predominately publicly insured population with a strong cultural proclivity for vaginal birth; Group C, private/academic maternal-fetal medicine (MFM) providers; Group D, midwifery providers serving patients mostly with public insurance; and Group E, the laborist-trainee group that also primarily serves patients with public insurance. Counseling regarding mode of delivery occurs during prenatal care by the same providers who ultimately deliver the patient. Our institutional standard of care requires patients with a single prior CD to sign a TOLAC consent form prenatally that must be re-attested at the time of admission for delivery.

Disparities between different obstetric provider practice groups were examined, with delivery outcome as the basis for analysis. Proportional differences of demographic and clinical characteristics were analyzed between different provider practice groups using Chi-squared tests. A two-sided p  < 0.05 was considered statistically significant. We examined the likelihood of successful versus unsuccessful VBAC using a pair of logistic regression models, with the multivariate model controlling for age, pre-pregnancy BMI, gestational age at delivery (in weeks), birthweight (in grams), history of prior vaginal delivery and VBAC, and indications for the primary CD. The statistical program R housed these analyses [ 22 ].

1,439 patients delivered a pregnancy after a previous cesarean during the study interval who met inclusion criteria. 993 patients had a successful VBAC (69%) while 205 patients had a SRCD (14%) and 241 had an unsuccessful VBAC resulting in CD (17%).

Table  1 displays demographic characteristics of the sample by provider practice type.

Age at delivery was significantly different between provider practice groups; 13% of patients in Group C were older than 40 years, compared to 2% of patients delivered by Group E ( p  < 0.001). Differences were noted by gestational age at delivery; 26% of Group C patients delivered before 37 weeks’ gestation compared to 10% of patients delivered by Group A. Conversely, 17% of patients from Group B delivered at > 40 weeks’ gestation, compared to 5% of Group C patients ( p  < 0.001). Different provider groups served patients at different levels of social vulnerability; 79% of patients delivered by Group E lived in high-vulnerability areas compared to 26% of patients delivered by Group A ( p  < 0.001). Provider practice groups differed significantly by race and ethnicity; 99% of patients in Group B identified as White compared to just 5% of patients delivered by Group E ( p  < 0.001). Patients in different practice groups had significantly different rates of private and public insurance; 74% of patients delivered by Group A had private insurance, compared to just 15% of patients delivered by Group D ( p  < 0.001). Pre-pregnancy BMI also differed significantly—14% of patients delivered by Group E had a pre-pregnancy BMI of Class III Obesity compared to 2% of patients delivered by Group A ( p  < 0.001). A significant proportion of subjects in all provider practice groups were missing third-trimester EFW data; for the available data, there were significant differences between provider practice groups ( p  < 0.01).

Table  2 shows the distribution of clinical characteristics by provider practice group.

Median gravidity was highest for Group B patients (6, IQR:6) and lowest for Groups A and D patients (3, IQR:2). Median term parity was highest for Group B patients (4, IQR:4) and lowest for Group A patients (1, IQR:0). Patient groups differed significantly by vaginal delivery history; 66% of Group B patients had a previous vaginal delivery compared to 17% of Group A patients ( p  < 0.001). 57% of Group B patients had a VBAC before the delivery indexed in the present study, compared to 14% of Group A patients ( p  < 0.001). There were also significant proportional differences with several primary cesarean indications as displayed in Table  2 .

Table  3 displays proportional differences in delivery outcomes among different provider practice groups.

There were significant proportional differences regarding mode of delivery ( p  < 0.001). 91% of patients delivered by Group B providers had a successful VBAC, compared to 53% of patients delivered by Group A. 29% of Group A patients delivered by SRCD, compared to zero patients delivered by Group D. 29% of patients delivered by Group E had an unsuccessful VBAC, compared to 1% of patients delivered by Group D. Rates of severe intrapartum complications (uterine rupture, hysterectomy, placental abruption) did not differ significantly between different provider practice groups. Neonates of subjects delivered by different provider practice groups differed significantly by admission to the Neonatal Intensive Care Unit (NICU), with a greater proportion of neonates from the Group D (15%) going to the NICU ( p  < 0.01). There were not adequate data on other neonatal outcomes for reporting in this paper.

Table  4 displays the results of two logistic models, one univariate and one multivariate regression, predicting odds ratios of successful VBAC versus unsuccessful VBAC.

The multivariate model adjusted for previously identified covariates. In the multivariate model, patients in Group D had a significantly greater odds ratio of successful VBAC compared to Group A (15.4; 95% CI: 3.2, 27.7). Patients delivered by Group E, by contrast, had a significantly lower odds ratio of successful VBAC (0.6, 95% CI: 0.4, 0.8).

This study reveals provider practice differences in delivery outcomes. While patients of Group B frequently chose TOLAC and delivered by VBAC, patients of Group A frequently chose SRCD. In multivariate models, patients delivered by Group D had significantly greater odds of successful VBAC versus unsuccessful VBAC compared to Group A. Patients delivered by Group E had significantly lower odds of successful VBAC compared to patients delivered by Group A.

Repeat CD was more prevalent in patients of private OBs, which is consistent prior data [ 14 , 16 ]. Rosenstein et al. compared VBAC rates between a private practice and a collaborative midwifery-laborist model and reported decreased VBAC among private OBs. With a combined midwifery-laborist system, the VBAC rate rose by 11% [ 15 ]. Metz et al. similarly found that patients of family practitioners more often selected TOLAC [ 12 ]. It should be noted that, at our institution, midwives do not perform CDs and their patients who require such intervention are delivered by the Laborist group. This therefore misrepresents the true rate of SRCD among patients in the midwifery group.

Importantly, different provider practice groups care for significantly different patient populations. These populations differ by demographic and clinical characteristics, as Tables  1 and 2 show. MFM specialists, for example, care for high-risk pregnancies, while midwives serve a lower risk population. For example, more than twice as many patients delivered preterm in the MFM group, likely iatrogenic in many cases. As revealed in this study, private OB patients are overwhelmingly insured by private health plans. Further, certain patient groups live in areas characterized by high social vulnerability, and this impacts prenatal care [ 23 ]. Finally, patients of different practices differed significantly across indications for their primary CD.

Prior studies have suggested associations between midwifery and TOLAC. Patients of midwives represent a self-selected population with preferences for vaginal birth [ 12 , 13 ]. The increased prevalence of TOLAC may reflect the philosophy of midwifery: women have the natural capacity to give birth without routine intervention [ 23 , 24 , 25 ]. A multicenter study found midwives were less likely to feel planned birth indications as “necessary” compared to other delivering providers [ 26 ]. Among both OBs and midwives, fear of malpractice litigation has clinical practice; one US survey found an incidence of litigation among midwives at 32%, while another indicated correlation between fear of litigation and CD rates [ 27 , 28 , 29 ]. Fear of litigation can influence provider choices regarding provision of care for their patients.

We found that TOLAC and VBAC were more common among certain provider groups, like Group B. This group comprises an OB practice that serves a patient population with strong cultural drivers for birth proliferation as well as avoidance of CD. We included this group to gain insight into cultural determinants of the delivery decision-making process. Only Pomeranz et al. have examined the relationship between patient-level cultural factors and delivery approach. Nearly one-third of their participants reported that providers had the strongest impact on their delivery method decision [ 20 ]. Further prospective investigation stands to illuminate these decision-making pathways [30].

Strengths of this study include the large, diverse cohort and minimal loss to follow-up. Overall, our sample was heterogeneous and included patients with varying probability of VBAC success. Some patients with one or multiple prior vaginal deliveries may have had greater odds of TOLAC and successful VBAC. Nevertheless, that was not our major area of focus as we were more concerned with provider-practice level differences and the overall process of decision making about the delivery approach.

Limitations include the fact that the study was retrospective and was based at a single institution, limiting generalizability. Examining different provider practice groups introduces complexity in that different providers provide slightly different care; this is apparent in the missing data for third-trimester EFW, where different provider practice groups varied significantly in the proportion of their patients that received a recorded third-trimester EFW ultrasound. Statistically, this study was limited in that some patient populations were ethnically homogenous or too small for sufficient comparison. This can be seen in the rates of VBAC among Groups B and D, exceeding 90%, which outpaces that reported and literature and is likely attributable to the homogeneous cultural drivers towards pursuit of VBAC in these patient populations. Another limitation relates to operationalizing provider practice type, determined based on authorship of the labor and delivery note in patients’ electronic medical records. Categorizing providers this way may not reflect which provider counseled patients. We cannot disentangle associations related to provider-level factors and patient-level factors, as patients may choose their providers, and in turn, their mode of delivery. Patients have personal preferences, and one can assume patients choose providers that align with those preferences when possible. There is therefore opportunity for selection bias. This is particularly true for Groups B and D. There is ample literature supporting the preference of midwife patients for TOLAC [ 12 , 13 , 23 , 24 , 25 ]. Thus, it is certainly possible that patients drove certain delivery patterns. Additionally, while midwives are equally involved in prenatal counseling, they are less involved in surgical practice, which introduces bias. There were limitations regarding sample size and distribution such that certain confidence intervals were imprecise in the multivariate models; a larger sample size would resolve this. Finally, we chose to group patients who received care from laborists and residents under one category (Group E), but there are likely differences between these providers.

Conclusions

Our results suggest significant disparities between different provider practice groups regarding mode of delivery for patients with a history of CD, in ways that appreciably affect VBAC success. These data also illuminate under-researched social drivers of health that influence maternal delivery outcomes. The findings from the present study contribute to ways in which the patient-provider interaction influences outcomes in obstetrics.

Data availability

Health data included in this study were abstracted from medical records electronically housed in our medical institution. Limited data sets are available by request to the corresponding author as allowed by institutionally governed data transfer agreements.

Abbreviations

Cesarean delivery

Obstetrician

Trial of labor after cesarean

Vaginal birth after cesarean

Scheduled repeat cesarean delivery

Maternal–fetal medicine

Health Insurance Portability and Accountability Act

Institutional review board

Program for the protection of human subjects

Body mass index

Spong CY, Berghella V, Wenstrom KD, Mercer BM, Saade GR. Preventing the first cesarean delivery: summary of a joint Eunice Kennedy Shriver National Institute of Child Health and Human Development, Society for Maternal-Fetal Medicine, and American College of Obstetricians and gynecologists Workshop. Obstet Gynecol. 2012;120(5):1181–93. https://doi.org/10.1097/aog.0b013e3182704880 . PMID: 23090537; PMCID: PMC3548444.

Article   PubMed   PubMed Central   Google Scholar  

Barber EL, Lundsberg LS, Belanger K, Pettker CM, Funai EF, Illuzzi JL. Indications contributing to the increasing cesarean delivery rate. Obstet Gynecol. 2011;118(1):29–38. https://doi.org/10.1097/AOG.0b013e31821e5f65 . PMID: 21646928; PMCID: PMC3751192.

ACOG Practice bulletin no. 115: Vaginal birth after previous cesarean delivery. Obstet Gynecol. 2010;116(2 Pt 1):450–463. https://doi.org/10.1097/AOG.0b013e3181eeb251 . PMID: 20664418.

Bălălău OD, Bacalbașa N, Olaru OG, Pleș L, Stănescu DA. Vaginal birth after cesarean section – literature review and modern guidelines. J Clin Invest Surg. 2020;5(1):13–7. https://doi.org/10.25083/2559.5555/5.1/13.17 .

Article   Google Scholar  

Qiu L, Zhu J, Lu X. The safety of trial of labor after cesarean section (TOLAC) versus elective repeat cesarean section (ERCS): a systematic review and meta-analysis. J Matern Fetal Neonatal Med. 2023;36(1):2214831. https://doi.org/10.1080/14767058.2023.2214831 . PMID: 37217450.

Hammad IA, Chauhan SP, Magann EF, Abuhamad AZ. Peripartum complications with cesarean delivery: a review of Maternal-Fetal Medicine Units Network publications. J Matern Fetal Neonatal Med. 2014;27(5):463 – 74. doi: 10.3109/14767058.2013.818970. Epub 2013 Sep 11. PMID: 24007280.

Markovic ES, Fox NS. Morbidity of repeat cesarean delivery after a trial of labor as compared with elective repeat cesarean delivery. Am J Perinatol. 2023. https://doi.org/10.1055/a-126-7613 .

Article   PubMed   Google Scholar  

Amikam U, Hochberg A, Segal R, Abramov S, Lavie A, Yogev Y, Hiersch L. Perinatal outcomes following uterine rupture during a trial of labor after cesarean: a 12-year single-center experience. Int J Gynecol Obstet. 2023;165(1):237–43. https://doi.org/10.1002/ijgo.15178 .

Delafield R, Elia J, Chang A, Kaneshiro B, Sentell T, Pirkle CM. Perspectives and experiences of obstetricians who provide Labor and Delivery Care for Micronesian women in Hawai‘i: what is driving Cesarean Delivery Rates? Qual Health Res. 2020;30(14):2291–302. https://doi.org/10.1177/1049732320942484 .

Sindiani A, Rawashdeh H, Obeidat N, Zayed F, Alhowary AA. Factors that influenced pregnant women with one previous cesarean section regarding their mode of delivery. Ann Med Surg. 2020;55:124–30. https://doi.org/10.1016/j.amsu.2020.05.007 .

Wingert A, Johnson C, Featherstone R, Sebastianski M, Hartling L, Douglas Wilson R. Adjunct clinical interventions that influence vaginal birth after cesarean rates: systematic review. BMC Pregnancy Childbirth. 2018;18(1):452. https://doi.org/10.1186/s12884-018-2065-x . PMID: 30463530; PMCID: PMC6249876.

Thirukumar P, Henry A, Coates D. Women’s experiences and involvement in decision-making in relation to planned cesarean birth: an interview study. J Perinat Edu. 2021;30(4). https://doi.org/10.1891/j-pe-d-20-00034 .

White HK, le May A, Cluett ER. Evaluating a midwife-led model of Antenatal Care for women with a previous cesarean section: a retrospective, comparative cohort study. Birth. 2016;43(3):200–8. https://doi.org/10.1111/birt.12229 . Epub 2016 Mar 18. PMID: 26991669.

Zhang T, Liu C. Comparison between continuing midwifery care and standard maternity care in vaginal birth after cesarean. Pak J Med Sci 2016 May-Jun;32(3):711–4. https://doi.org/10.12669/pjms.323.9546 . PMID: 27375719; PMCID: PMC4928428.

Rosenstein MG, Nijagal M, Nakagawa S, Gregorich SE, Kuppermann M. The Association of Expanded Access to a collaborative midwifery and Laborist Model with Cesarean Delivery Rates. Obstet Gynecol. 2015;126(4):716–23. https://doi.org/10.1097/AOG.0000000000001032 . PMID: 26348175; PMCID: PMC4580519.

Nijagal MA, Kuppermann M, Nakagawa S, Cheng Y. Two practice models in one labor and delivery unit: association with cesarean delivery rates. Am J Obstet Gynecol. 2015;212(4):491.e1-8. https://doi.org/10.1016/j.ajog.2014.11.014 . Epub 2014 Nov 13. Erratum in: Am J Obstet Gynecol. 2015;213(3):400. PMID: 25446697; PMCID: PMC4387106.

Feldman DS, Bollman DL, Fridman M, Korst LM, El Haj Ibrahim S, Fink A, Gregory KD. Do laborists improve delivery outcomes for laboring women in California community hospitals? Am J Obstet Gynecol. 2015;213(4):587.e1-587.e13. doi: 10.1016/j.ajog.2015.05.051. Epub 2015 May 28. PMID: 26026921.

Yee LM, Liu LY, Grobman WA. Obstetrician call schedule and obstetric outcomes among women eligible for a trial of labor after cesarean. Am J Obstet Gynecol. 2017;216(1):75. https://doi.org/10.1016/j.ajog.2016.08.032 . Epub 2016 Aug 30. PMID: 27589899; PMCID: PMC5182154.

Johansson M, Alvan J, Hildingsson I. Conflicting attitudes between clinicians and women regarding maternal requested caesarean section: a qualitative evidence synthesis. BCM Pregnancy Childbirth. 2023;23(210). https://doi.org/10.1186/s12884-023-05471-2 .

Pomeranz M, Arbib N, Haddif L, Reissner H, Romem Y, Biron T. In God we trust and other factors influencing trial of labor versus repeat cesarean section. J Matern Fetal Neonatal Med. 2018;31(13):1777–81. Epub 2017 May 24. PMID: 28475396.

Flatt S, Velez MP. The cost of preterm birth and cesarean section as a result of infertility and its treatment: a review. Best Pract Res Clin Obstet Gynecol. 2023;86. https://doi.org/10.1016/j.bpobgyn.2022.102304 .

Perez MT, Bucholz E, Asimacopoulos E, Ferraro AM, Salem SM, Schauer J, Hollerman C, Sekhavat S, Tworetzky W, Powell AJ, Sleeper LA, Beroukhim RS. Impact of maternal social vulnerability and timing of prenatal care on outcome of prenatally detected congenital heart disease. Ultrasound Ob Gyn. 2022;60(3):346–58. https://doi.org/10.1002/uog.24863 .

Article   CAS   Google Scholar  

Sandall J, Soltani H, Gates S, Shennan A, Devane D. Midwife-led continuity models versus other models of care for childbearing women. Cochrane Database Syst Rev. 2015;(9):CD004667. https://doi.org/10.1002/14651858.CD004667.pub4 . Update in: Cochrane Database Syst Rev. 2016;4:CD004667. PMID: 26370160.

Care for women desiring vaginal birth after cesarean: American College of Nurse-Midwives. J Midwifery Womens Health. 2011 Sep-Oct;56(5):517 – 25. https://doi.org/10.1111/j.1542-2011.2011.00112.x . PMID: 23181655.

Coates D, Donnolley N, Foureur M, Spear V, Henry A. Exploring unwarranted clinical variation: the attitudes of midwives and obstetric medical staff regarding induction of labour and planned caesarean section. Women Birth. 2021;34(4):351–61. https://doi.org/10.1016/j.wombi.2020.07.003 .

Lundgren I, van Limbeek E, Vehvilainen-Julkunen K, Nilsson C. Clinicians’ views of factors of importance for improving the rate of VBAC (vaginal birth after caesarean section): a qualitative study from countries with high VBAC rates. BMC Pregnancy Childbirth. 2015;15:196. https://doi.org/10.1186/s12884-015-0629-6 . PMID: 26314295; PMCID: PMC4552403.

Page K, Aryal S, Guidera M. Midwives and liability: results of the 2018 National midwives and Liability Survey. J Midwifery Women’s Health. 2022;67(2):226–34.

Minkoff H. Fear of litigation and cesarean section rates. Semin Perinatol. Oct; 2012;36(5):390–4. https://doi.org/10.1053/j.semperi.2012.04.025 .

Perrotta C, Romero M, Sguassero Y, Straw C, Gialdini C, Righetti N, Betran AP, Ramos S. Caesarean birth in public maternities in Argentina: a formative research study on the views of obstetricians, midwives, and trainees. BMJ 2021; 12(1).

Download references

Acknowledgements

We would like to thank the study team who started this investigation, particularly Drs. Ayisha Buckley, Luciana Vieira, and Joanne Stone, for pushing the boundaries of how we understand labor and delivery after prior cesarean.

This study was not directly supported by any funding; however, the Department of Obstetrics, Gynecology, and Reproductive Science at the Icahn School of Medicine at Mount Sinai supported the salaries of individuals who worked on this project. Additionally, an institutional grant from the National Center for Advancing Translational Sciences facilitated the data capture tools used by this study.

Author information

Authors and affiliations.

Graduate School of Biomedical Sciences, Icahn School of Medicine at Mount Sinai Hospital, New York, NY, 10029, USA

Lily McCarthy, Sara Wetzler & Lily Ardente

Department of Obstetrics, Gynecology, and Reproductive Science, Icahn School of Medicine at Mount Sinai Hospital, New York, NY, 10029, USA

Nicola F Tavella, Dexter Paul, Nikki Sabet, Toni Stern & Angela Bianco

Princeton University, Princeton, NJ, 08544, USA

Molly Chadwick

Maternal-Fetal Medicine, Icahn School of Medicine at Mount Sinai Hospital, New York, NY, 10029, USA

Nicola F Tavella & Angela Bianco

You can also search for this author in PubMed   Google Scholar

Contributions

Study conceived and designed by LM, NFT, SW, LA, TS, AB. Data collected by LM, NFT, SW, LA, MC, DP, NS. Data analyzed by NFT. Manuscript developed and reviewed by LM, NFT, SW, TS, AB.

Corresponding author

Correspondence to Nicola F Tavella .

Ethics declarations

Ethics approval and consent to participate.

This study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board and the Program for the Protection of Human Subjects at the Icahn School of Medicine at Mount Sinai on 17 June 2020 under the study ID number 20–00633. The Institutional Review Board at the Icahn School of Medicine at Mount Sinai granted a waiver of informed consent and HIPAA to conduct this retrospective study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

McCarthy, L., Tavella, N.F., Wetzler, S. et al. Examining provider practice-level disparities in delivery outcomes among patients with a history of Cesarean Delivery. BMC Pregnancy Childbirth 24 , 243 (2024). https://doi.org/10.1186/s12884-024-06458-3

Download citation

Received : 18 January 2024

Accepted : 27 March 2024

Published : 05 April 2024

DOI : https://doi.org/10.1186/s12884-024-06458-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Trial of labor
• Retrospective studies
• Multivariate analysis

BMC Pregnancy and Childbirth

ISSN: 1471-2393

• Submission enquiries: [email protected]
• General enquiries: [email protected]