multiple case studies research design

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Research Guides

Multiple Case Studies

Nadia Alqahtani and Pengtong Qu

Description

The case study approach is popular across disciplines in education, anthropology, sociology, psychology, medicine, law, and political science (Creswell, 2013). It is both a research method and a strategy (Creswell, 2013; Yin, 2017). In this type of research design, a case can be an individual, an event, or an entity, as determined by the research questions. There are two variants of the case study: the single-case study and the multiple-case study. The former design can be used to study and understand an unusual case, a critical case, a longitudinal case, or a revelatory case. On the other hand, a multiple-case study includes two or more cases or replications across the cases to investigate the same phenomena (Lewis-Beck, Bryman & Liao, 2003; Yin, 2017). …a multiple-case study includes two or more cases or replications across the cases to investigate the same phenomena

The difference between the single- and multiple-case study is the research design; however, they are within the same methodological framework (Yin, 2017). Multiple cases are selected so that “individual case studies either (a) predict similar results (a literal replication) or (b) predict contrasting results but for anticipatable reasons (a theoretical replication)” (p. 55). When the purpose of the study is to compare and replicate the findings, the multiple-case study produces more compelling evidence so that the study is considered more robust than the single-case study (Yin, 2017).

To write a multiple-case study, a summary of individual cases should be reported, and researchers need to draw cross-case conclusions and form a cross-case report (Yin, 2017). With evidence from multiple cases, researchers may have generalizable findings and develop theories (Lewis-Beck, Bryman & Liao, 2003).

Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). Los Angeles, CA: Sage.

Lewis-Beck, M., Bryman, A. E., & Liao, T. F. (2003). The Sage encyclopedia of social science research methods . Los Angeles, CA: Sage.

Yin, R. K. (2017). Case study research and applications: Design and methods . Los Angeles, CA: Sage.

Key Research Books and Articles on Multiple Case Study Methodology

Yin discusses how to decide if a case study should be used in research. Novice researchers can learn about research design, data collection, and data analysis of different types of case studies, as well as writing a case study report.

Chapter 2 introduces four major types of research design in case studies: holistic single-case design, embedded single-case design, holistic multiple-case design, and embedded multiple-case design. Novice researchers will learn about the definitions and characteristics of different designs. This chapter also teaches researchers how to examine and discuss the reliability and validity of the designs.

Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing among five approaches . Los Angeles, CA: Sage.

This book compares five different qualitative research designs: narrative research, phenomenology, grounded theory, ethnography, and case study. It compares the characteristics, data collection, data analysis and representation, validity, and writing-up procedures among five inquiry approaches using texts with tables. For each approach, the author introduced the definition, features, types, and procedures and contextualized these components in a study, which was conducted through the same method. Each chapter ends with a list of relevant readings of each inquiry approach.

This book invites readers to compare these five qualitative methods and see the value of each approach. Readers can consider which approach would serve for their research contexts and questions, as well as how to design their research and conduct the data analysis based on their choice of research method.

Günes, E., & Bahçivan, E. (2016). A multiple case study of preservice science teachers’ TPACK: Embedded in a comprehensive belief system. International Journal of Environmental and Science Education, 11 (15), 8040-8054.

In this article, the researchers showed the importance of using technological opportunities in improving the education process and how they enhanced the students’ learning in science education. The study examined the connection between “Technological Pedagogical Content Knowledge” (TPACK) and belief system in a science teaching context. The researchers used the multiple-case study to explore the effect of TPACK on the preservice science teachers’ (PST) beliefs on their TPACK level. The participants were three teachers with the low, medium, and high level of TPACK confidence. Content analysis was utilized to analyze the data, which were collected by individual semi-structured interviews with the participants about their lesson plans. The study first discussed each case, then compared features and relations across cases. The researchers found that there was a positive relationship between PST’s TPACK confidence and TPACK level; when PST had higher TPACK confidence, the participant had a higher competent TPACK level and vice versa.

Recent Dissertations Using Multiple Case Study Methodology

Milholland, E. S. (2015). A multiple case study of instructors utilizing Classroom Response Systems (CRS) to achieve pedagogical goals . Retrieved from ProQuest Dissertations & Theses Global. (Order Number 3706380)

The researcher of this study critiques the use of Classroom Responses Systems by five instructors who employed this program five years ago in their classrooms. The researcher conducted the multiple-case study methodology and categorized themes. He interviewed each instructor with questions about their initial pedagogical goals, the changes in pedagogy during teaching, and the teaching techniques individuals used while practicing the CRS. The researcher used the multiple-case study with five instructors. He found that all instructors changed their goals during employing CRS; they decided to reduce the time of lecturing and to spend more time engaging students in interactive activities. This study also demonstrated that CRS was useful for the instructors to achieve multiple learning goals; all the instructors provided examples of the positive aspect of implementing CRS in their classrooms.

Li, C. L. (2010). The emergence of fairy tale literacy: A multiple case study on promoting critical literacy of children through a juxtaposed reading of classic fairy tales and their contemporary disruptive variants . Retrieved from ProQuest Dissertations & Theses Global. (Order Number 3572104)

To explore how children’s development of critical literacy can be impacted by their reactions to fairy tales, the author conducted a multiple-case study with 4 cases, in which each child was a unit of analysis. Two Chinese immigrant children (a boy and a girl) and two American children (a boy and a girl) at the second or third grade were recruited in the study. The data were collected through interviews, discussions on fairy tales, and drawing pictures. The analysis was conducted within both individual cases and cross cases. Across four cases, the researcher found that the young children’s’ knowledge of traditional fairy tales was built upon mass-media based adaptations. The children believed that the representations on mass-media were the original stories, even though fairy tales are included in the elementary school curriculum. The author also found that introducing classic versions of fairy tales increased children’s knowledge in the genre’s origin, which would benefit their understanding of the genre. She argued that introducing fairy tales can be the first step to promote children’s development of critical literacy.

Asher, K. C. (2014). Mediating occupational socialization and occupational individuation in teacher education: A multiple case study of five elementary pre-service student teachers . Retrieved from ProQuest Dissertations & Theses Global. (Order Number 3671989)

This study portrayed five pre-service teachers’ teaching experience in their student teaching phase and explored how pre-service teachers mediate their occupational socialization with occupational individuation. The study used the multiple-case study design and recruited five pre-service teachers from a Midwestern university as five cases. Qualitative data were collected through interviews, classroom observations, and field notes. The author implemented the case study analysis and found five strategies that the participants used to mediate occupational socialization with occupational individuation. These strategies were: 1) hindering from practicing their beliefs, 2) mimicking the styles of supervising teachers, 3) teaching in the ways in alignment with school’s existing practice, 4) enacting their own ideas, and 5) integrating and balancing occupational socialization and occupational individuation. The study also provided recommendations and implications to policymakers and educators in teacher education so that pre-service teachers can be better supported.

Multiple Case Studies Copyright © 2019 by Nadia Alqahtani and Pengtong Qu is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Browse Econ Literature

• Working papers
• Software components
• Book chapters
• JEL classification

More features

• Subscribe to new research

RePEc Biblio

Author registration.

• Economics Virtual Seminar Calendar NEW!

Multiple Case Research Design

In: research design in business and management.

• Author & abstract
• Related works & more

Corrections

(Hochschule Luzern)

Suggested Citation

Download full text from publisher.

Follow serials, authors, keywords & more

Public profiles for Economics researchers

Various research rankings in Economics

RePEc Genealogy

Who was a student of whom, using RePEc

Curated articles & papers on economics topics

Upload your paper to be listed on RePEc and IDEAS

New papers by email

Subscribe to new additions to RePEc

EconAcademics

Blog aggregator for economics research

Cases of plagiarism in Economics

About RePEc

Initiative for open bibliographies in Economics

News about RePEc

Questions about IDEAS and RePEc

RePEc volunteers

Participating archives

Publishers indexing in RePEc

Privacy statement

Found an error or omission?

Opportunities to help RePEc

Get papers listed

Have your research listed on RePEc

Open a RePEc archive

Have your institution's/publisher's output listed on RePEc

Qualitative Research Designs

Case study design, using case study design in the applied doctoral experience (ade), applicability of case study design to applied problem of practice, case study design references.

• SAGE Research Methods
• Research Examples (SAGE) This link opens in a new window
• Dataset Examples (SAGE) This link opens in a new window
• IRB Resource Center This link opens in a new window

The field of qualitative research there are a number of research designs (also referred to as “traditions” or “genres”), including case study, phenomenology, narrative inquiry, action research, ethnography, grounded theory, as well as a number of critical genres including Feminist theory, indigenous research, critical race theory and cultural studies. The choice of research design is directly tied to and must be aligned with your research problem and purpose. As Bloomberg & Volpe (2019) explain:

Choice of research design is directly tied to research problem and purpose. As the researcher, you actively create the link among problem, purpose, and design through a process of reflecting on problem and purpose, focusing on researchable questions, and considering how to best address these questions. Thinking along these lines affords a research study methodological congruence (p. 38).

Case study is an in-depth exploration from multiple perspectives of a bounded social phenomenon, be this a social system such as a program, event, institution, organization, or community (Stake, 1995, 2005; Yin, 2018). Case study is employed across disciplines, including education, health care, social work, sociology, and organizational studies. The purpose is to generate understanding and deep insights to inform professional practice, policy development, and community or social action (Bloomberg 2018).

Yin (2018) and Stake (1995, 2005), two of the key proponents of case study methodology, use different terms to describe case studies. Yin categorizes case studies as exploratory or descriptive . The former is used to explore those situations in which the intervention being evaluated has no clear single set of outcomes. The latter is used to describe an intervention or phenomenon and the real-life context in which it occurred. Stake identifies case studies as intrinsic or instrumental , and he proposes that a primary distinction in designing case studies is between single and multiple (or collective) case study designs. A single case study may be an instrumental case study (research focuses on an issue or concern in one bounded case) or an intrinsic case study (the focus is on the case itself because the case presents a unique situation). A longitudinal case study design is chosen when the researcher seeks to examine the same single case at two or more different points in time or to capture trends over time. A multiple case study design is used when a researcher seeks to determine the prevalence or frequency of a particular phenomenon. This approach is useful when cases are used for purposes of a cross-case analysis in order to compare, contrast, and synthesize perspectives regarding the same issue. The focus is on the analysis of diverse cases to determine how these confirm the findings within or between cases, or call the findings into question.

Case study affords significant interaction with research participants, providing an in-depth picture of the phenomenon (Bloomberg & Volpe, 2019). Research is extensive, drawing on multiple methods of data collection, and involves multiple data sources. Triangulation is critical in attempting to obtain an in-depth understanding of the phenomenon under study and adds rigor, breadth, and depth to the study and provides corroborative evidence of the data obtained. Analysis of data can be holistic or embedded—that is, dealing with the whole or parts of the case (Yin, 2018). With multiple cases the typical analytic strategy is to provide detailed description of themes within each case (within-case analysis), followed by thematic analysis across cases (cross-case analysis), providing insights regarding how individual cases are comparable along important dimensions. Research culminates in the production of a detailed description of a setting and its participants, accompanied by an analysis of the data for themes or patterns (Stake, 1995, 2005; Yin, 2018). In addition to thick, rich description, the researcher’s interpretations, conclusions, and recommendations contribute to the reader’s overall understanding of the case study.

Analysis of findings should show that the researcher has attended to all the data, should address the most significant aspects of the case, and should demonstrate familiarity with the prevailing thinking and discourse about the topic. The goal of case study design (as with all qualitative designs) is not generalizability but rather transferability —that is, how (if at all) and in what ways understanding and knowledge can be applied in similar contexts and settings. The qualitative researcher attempts to address the issue of transferability by way of thick, rich description that will provide the basis for a case or cases to have relevance and potential application across a broader context.

Qualitative research methods ask the questions of "what" and "how" a phenomenon is understood in a real-life context (Bloomberg & Volpe, 2019). In the education field, qualitative research methods uncover educational experiences and practices because qualitative research allows the researcher to reveal new knowledge and understanding. Moreover, qualitative descriptive case studies describe, analyze and interpret events that explain the reasoning behind specific phenomena (Bloomberg, 2018). As such, case study design can be the foundation for a rigorous study within the Applied Doctoral Experience (ADE).

Case study design is an appropriate research design to consider when conceptualizing and conducting a dissertation research study that is based on an applied problem of practice with inherent real-life educational implications. Case study researchers study current, real-life cases that are in progress so that they can gather accurate information that is current. This fits well with the ADE program, as students are typically exploring a problem of practice. Because of the flexibility of the methods used, a descriptive design provides the researcher with the opportunity to choose data collection methods that are best suited to a practice-based research purpose, and can include individual interviews, focus groups, observation, surveys, and critical incident questionnaires. Methods are triangulated to contribute to the study’s trustworthiness. In selecting the set of data collection methods, it is important that the researcher carefully consider the alignment between research questions and the type of data that is needed to address these. Each data source is one piece of the “puzzle,” that contributes to the researcher’s holistic understanding of a phenomenon. The various strands of data are woven together holistically to promote a deeper understanding of the case and its application to an educationally-based problem of practice.

Research studies within the Applied Doctoral Experience (ADE) will be practical in nature and focus on problems and issues that inform educational practice.  Many of the types of studies that fall within the ADE framework are exploratory, and align with case study design. Case study design fits very well with applied problems related to educational practice, as the following set of examples illustrate:

Elementary Bilingual Education Teachers’ Self-Efficacy in Teaching English Language Learners: A Qualitative Case Study

The problem to be addressed in the proposed study is that some elementary bilingual education teachers’ beliefs about their lack of preparedness to teach the English language may negatively impact the language proficiency skills of Hispanic ELLs (Ernst-Slavit & Wenger, 2016; Fuchs et al., 2018; Hoque, 2016). The purpose of the proposed qualitative descriptive case study was to explore the perspectives and experiences of elementary bilingual education teachers regarding their perceived lack of preparedness to teach the English language and how this may impact the language proficiency of Hispanic ELLs.

Exploring Minority Teachers Experiences Pertaining to their Value in Education: A Single Case Study of Teachers in New York City

The problem is that minority K-12 teachers are underrepresented in the United States, with research indicating that school leaders and teachers in schools that are populated mainly by black students, staffed mostly by white teachers who may be unprepared to deal with biases and stereotypes that are ingrained in schools (Egalite, Kisida, & Winters, 2015; Milligan & Howley, 2015). The purpose of this qualitative exploratory single case study was to develop a clearer understanding of minority teachers’ experiences concerning the under-representation of minority K-12 teachers in urban school districts in the United States since there are so few of them.

Exploring the Impact of an Urban Teacher Residency Program on Teachers’ Cultural Intelligence: A Qualitative Case Study

The problem to be addressed by this case study is that teacher candidates often report being unprepared and ill-equipped to effectively educate culturally diverse students (Skepple, 2015; Beutel, 2018). The purpose of this study was to explore and gain an in-depth understanding of the perceived impact of an urban teacher residency program in urban Iowa on teachers’ cultural competence using the cultural intelligence (CQ) framework (Earley & Ang, 2003).

Qualitative Case Study that Explores Self-Efficacy and Mentorship on Women in Academic Administrative Leadership Roles

The problem was that female school-level administrators might be less likely to experience mentorship, thereby potentially decreasing their self-efficacy (Bing & Smith, 2019; Brown, 2020; Grant, 2021). The purpose of this case study was to determine to what extent female school-level administrators in the United States who had a mentor have a sense of self-efficacy and to examine the relationship between mentorship and self-efficacy.

Suburban Teacher and Administrator Perceptions of Culturally Responsive Teaching to Promote Connectedness in Students of Color: A Qualitative Case Study

The problem to be addressed in this study is the racial discrimination experienced by students of color in suburban schools and the resulting negative school experience (Jara & Bloomsbury, 2020; Jones, 2019; Kohli et al., 2017; Wandix-White, 2020). The purpose of this case study is to explore how culturally responsive practices can counteract systemic racism and discrimination in suburban schools thereby meeting the needs of students of color by creating positive learning experiences. 

As you can see, all of these studies were well suited to qualitative case study design. In each of these studies, the applied research problem and research purpose were clearly grounded in educational practice as well as directly aligned with qualitative case study methodology. In the Applied Doctoral Experience (ADE), you will be focused on addressing or resolving an educationally relevant research problem of practice. As such, your case study, with clear boundaries, will be one that centers on a real-life authentic problem in your field of practice that you believe is in need of resolution or improvement, and that the outcome thereof will be educationally valuable.

Bloomberg, L. D. (2018). Case study method. In B. B. Frey (Ed.), The SAGE Encyclopedia of educational research, measurement, and evaluation (pp. 237–239). SAGE. https://go.openathens.net/redirector/nu.edu?url=https%3A%2F%2Fmethods.sagepub.com%2FReference%2Fthe-sage-encyclopedia-of-educational-research-measurement-and-evaluation%2Fi4294.xml

Bloomberg, L. D. & Volpe, M. (2019). Completing your qualitative dissertation: A road map from beginning to end . (4th Ed.). SAGE.

Stake, R. E. (1995). The art of case study research. SAGE.

Stake, R. E. (2005). Qualitative case studies. In N. K. Denzin and Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 443–466). SAGE.

Yin, R. (2018). Case study research and applications: Designs and methods. SAGE.

• << Previous: Action Research Resource
• Next: SAGE Research Methods >>
• Last Updated: Jul 28, 2023 8:05 AM
• URL: https://resources.nu.edu/c.php?g=1013605

• Open access
• Published: 16 August 2022

Developing an implementation research logic model: using a multiple case study design to establish a worked exemplar

• Louise Czosnek   ORCID: orcid.org/0000-0002-2362-6888 1 ,
• Eva M. Zopf 1 , 2 ,
• Prue Cormie 3 , 4 ,
• Simon Rosenbaum 5 , 6 ,
• Justin Richards 7 &
• Nicole M. Rankin 8 , 9  

Implementation Science Communications volume  3 , Article number:  90 ( 2022 ) Cite this article

7631 Accesses

3 Citations

21 Altmetric

Metrics details

Implementation science frameworks explore, interpret, and evaluate different components of the implementation process. By using a program logic approach, implementation frameworks with different purposes can be combined to detail complex interactions. The Implementation Research Logic Model (IRLM) facilitates the development of causal pathways and mechanisms that enable implementation. Critical elements of the IRLM vary across different study designs, and its applicability to synthesizing findings across settings is also under-explored. The dual purpose of this study is to develop an IRLM from an implementation research study that used case study methodology and to demonstrate the utility of the IRLM to synthesize findings across case sites.

The method used in the exemplar project and the alignment of the IRLM to case study methodology are described. Cases were purposely selected using replication logic and represent organizations that have embedded exercise in routine care for people with cancer or mental illness. Four data sources were selected: semi-structured interviews with purposely selected staff, organizational document review, observations, and a survey using the Program Sustainability Assessment Tool (PSAT). Framework analysis was used, and an IRLM was produced at each case site. Similar elements within the individual IRLM were identified, extracted, and re-produced to synthesize findings across sites and represent the generalized, cross-case findings.

The IRLM was embedded within multiple stages of the study, including data collection, analysis, and reporting transparency. Between 33-44 determinants and 36-44 implementation strategies were identified at sites that informed individual IRLMs. An example of generalized findings describing “intervention adaptability” demonstrated similarities in determinant detail and mechanisms of implementation strategies across sites. However, different strategies were applied to address similar determinants. Dependent and bi-directional relationships operated along the causal pathway that influenced implementation outcomes.

Conclusions

Case study methods help address implementation research priorities, including developing causal pathways and mechanisms. Embedding the IRLM within the case study approach provided structure and added to the transparency and replicability of the study. Identifying the similar elements across sites helped synthesize findings and give a general explanation of the implementation process. Detailing the methods provides an example for replication that can build generalizable knowledge in implementation research.

Peer Review reports

Contributions to the literature

Logic models can help understand how and why evidence-based interventions (EBIs) work to produce intended outcomes.

The implementation research logic model (IRLM) provides a method to understand causal pathways, including determinants, implementation strategies, mechanisms, and implementation outcomes.

We describe an exemplar project using a multiple case study design that embeds the IRLM at multiple stages. The exemplar explains how the IRLM helped synthesize findings across sites by identifying the common elements within the causal pathway.

By detailing the exemplar methods, we offer insights into how this approach of using the IRLM is generalizable and can be replicated in other studies.

The practice of implementation aims to get “someone…, somewhere… to do something differently” [ 1 ]. Typically, this involves changing individual behaviors and organizational processes to improve the use of evidence-based interventions (EBIs). To understand this change, implementation science applies different theories, models, and frameworks (hereafter “frameworks”) to describe and evaluate the factors and steps in the implementation process [ 2 , 3 , 4 , 5 ]. Implementation science provides much-needed theoretical frameworks and a structured approach to process evaluations. One or more frameworks are often used within a program of work to investigate the different stages and elements of implementation [ 6 ]. Researchers have acknowledged that the dynamic implementation process could benefit from using logic models [ 7 ]. Logic models offer a systematic approach to combining multiple frameworks and to building causal pathways that explain the mechanisms behind individual and organizational change.

Logic models visually represent how an EBI is intended to work [ 8 ]. They link the available resources with the activities undertaken, the immediate outputs of this work, and the intermediate outcomes and longer-term impacts [ 8 , 9 ]. Through this process, causal pathways are identified. For implementation research, the causal pathway provides the interconnection between a chosen EBI, determinants, implementation strategies, and implementation outcomes [ 10 ]. Testing causal mechanisms in the research translation pathway will likely dominate the next wave of implementation research [ 11 , 12 ]. Causal mechanisms (or mechanisms of change) are the “process or event through which an implementation strategy operates to affect desired implementation outcomes” [ 13 ]. Identifying mechanisms can improve implementation strategies’ selection, prioritization, and targeting [ 12 , 13 ]. This provides an efficient and evidence-informed approach to implementation.

Implementation researchers have proposed several methods to develop and examine causal pathways [ 14 , 15 ] and mechanisms [ 16 , 17 ]. This includes formalizing the inherent relationship between frameworks via developing the Implementation Research Logic Model (IRLM) [ 7 ]. The IRLM is a logic model designed to improve the rigor and reproducibility of implementation research. It specifies the relationship between elements of implementation (determinant, strategies, and outcomes) and the mechanisms of change. To do this, it recommends linking implementation frameworks or relevant taxonomies (e.g., determinant and evaluation frameworks and implementation strategy taxonomy). The IRLM authors suggest the tool has multiple uses, including planning, executing, and reporting on the implementation process and synthesizing implementation findings across different contexts [ 7 ]. During its development, the IRLM was tested to confirm its utility in planning, executing, and reporting; however, its utility in synthesizing findings across different contexts is ongoing. Users of the tool are encouraged to consider three principles: (1) comprehensiveness in reporting determinants, implementation strategies, and implementation outcomes; (2) specifying the conceptual relationships via diagrammatic tools such as colors and arrows; and (3) detailing important elements of the study design. Further, the authors also recognize that critical elements of IRLM will vary across different study designs.

This study describes the development of an IRLM from a multiple case study design. Case study methodology can answer “how and why” questions about implementation. They enable researchers to develop a rich, in-depth understanding of a contemporary phenomenon within its natural context [ 18 , 19 , 20 , 21 ]. These methods can create coherence in the dynamic context in which EBIs exist [ 22 , 23 ]. Case studies are common in implementation research [ 24 , 25 , 26 , 27 , 28 , 29 , 30 ], with multiple case study designs suitable for undertaking comparisons across contexts [ 31 , 32 ]. However, they are infrequently applied to establish mechanisms [ 11 ] or combine implementation elements to synthesize findings across contexts (as possible through the IRLM). Hollick and colleagues [ 33 ] undertook a comparative case study, guided by a determinant framework, to explore how context influences successful implementation. The authors contrasted determinants across sites where implementation was successful versus sites where implementation failed. The study did not extend to identifying implementation strategies or mechanisms. By contrast, van Zelm et al. [ 31 ] undertook a theory-driven evaluation of successful implementation across ten hospitals. They used joint displays to present mechanisms of change aligned with evaluation outcomes; however, they did not identify the implementation strategies within the causal pathway. Our study seeks to build on these works and explore the utility of the IRLM in synthesizing findings across sites. The dual objectives of this paper were to:

Describe how case study methods can be applied to develop an IRLM

Demonstrate the utility of the IRLM in synthesizing implementation findings across case sites.

In this section, we describe the methods used in the exemplar case study and the alignment of the IRLM to this approach. The exemplar study explored the implementation of exercise EBIs in the context of the Australian healthcare system. The exemplar study aimed to investigate the integration of exercise EBIs within routine mental illness or cancer care. The evidence base detailing the therapeutic benefits of exercise for non-communicable diseases such as cancer and mental illness are extensively documented [ 34 , 35 , 36 ] but inconsistently implemented as part of routine care [ 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 ].

Additional file 1 provides the Standards for Reporting Qualitative Research (SRQR).

Case study approach

We adopted an approach to case studies based on the methods described by Yin [ 18 ]. This approach is said to have post-positivist philosophical leanings, which are typically associated with the quantitative paradigm [ 19 , 45 , 46 ]. This is evidenced by the structured, deductive approach to the methods that are described with a constant lens on objectivity, validity, and generalization [ 46 ]. Yin’s approach to case studies aligns with the IRLM for several reasons. The IRLM is designed to use established implementation frameworks. The two frameworks and one taxonomy applied in our exemplar were the Consolidated Framework for Implementation Research (CFIR) [ 47 ], Expert Recommendations for Implementing Change (ERIC) [ 48 ], and Proctor et al.’s implementation outcomes framework [ 49 ]. These frameworks guided multiple aspects of our study (see Table 1 ). Commencing an implementation study with a preconceived plan based upon established frameworks is deductive [ 22 ]. Second, the IRLM has its foundation in logic modeling to develop cause and effect relationships [ 8 ]. Yin advocates using logic models to analyze case study findings [ 18 ]. They argue that developing logic models encourages researchers to iterate and consider plausible counterfactual explanations before upholding the causal pathway. Further, Yin notes that case studies are particularly valuable for explaining the transitions and context within the cause-and-effect relationship [ 18 ]. In our exemplar, the transition was the mechanism between the implementation strategy and implementation outcome. Finally, the proposed function of IRLM to synthesize findings across sites aligns with the exemplar study that used a multiple case approach. Multiple case studies aim to develop generalizable knowledge [ 18 , 50 ].

Case study selection and boundaries

A unique feature of Yin’s approach to multiple case studies is using replication logic to select cases [ 18 ]. Cases are chosen to demonstrate similarities (literal replication) or differences for anticipated reasons (theoretical replication) [ 18 ]. In the exemplar study, the cases were purposely selected using literal replication and displayed several common characteristics. First, all cases had delivered exercise EBIs within normal operations for at least 12 months. Second, each case site delivered exercise EBIs as part of routine care for a non-communicable disease (cancer or mental illness diagnosis). Finally, each site delivered the exercise EBI within the existing governance structures of the Australian healthcare system. That is, the organizations used established funding and service delivery models of the Australian healthcare system.

Using replication logic, we posited that sites would exhibit some similarities in the implementation process across contexts (literal replication). However, based on existing implementation literature [ 32 , 51 , 52 , 53 ], we expected sites to adapt the EBIs through the implementation process. The determinant analysis should explain these adaptions, which is informed by the CFIR (theoretical replication). Finally, in case study methods, clearly defining the boundaries of each case and the units of analysis, such as individual, the organization or intervention, helps focus the research. We considered each healthcare organization as a separate case. Within that, organizational-level analysis [ 18 , 54 ] and operationalizing the implementation outcomes focused inquiry (Table 1 ).

Data collection

During the study conceptualization for the exemplar, we mapped the data sources to the different elements of the IRLM (Fig. 1 ). Four primary data sources informed data collection: (1) semi-structured interviews with staff; (2) document review (such as meeting minutes, strategic plans, and consultant reports); (3) naturalistic observations; and (4) a validated survey (Program Sustainability Assessment Tool (PSAT)). A case study database was developed using Microsoft Excel to manage and organize data collection [ 18 , 54 ].

Conceptual frame for the study

Semi-structured interviews

An interview guide was developed, informed by the CFIR interview guide tool [ 55 ]. Questions were selected across the five domains of the CFIR, which aligned with the delineation of determinant domains in the IRLM. Purposeful selection was used to identify staff for the interviews [ 56 ]. Adequate sample size in qualitative studies, particularly regarding the number of interviews, is often determined when data saturation is reached [ 57 , 58 ]. Unfortunately, there is little consensus on the definition of saturation [ 59 ], how to interpret when it has occurred [ 57 ], or whether it is possible to pre-determine in qualitative studies [ 60 ]. The number of participants in this study was determined based on the staff’s differential experience with the exercise EBI and their role in the organization. This approach sought to obtain a rounded view of how the EBI operated at each site [ 23 , 61 ]. Focusing on staff experiences also aligned with the organizational lens that bounded the study. Typical roles identified for the semi-structured interviews included the health professional delivering the EBI, the program manager responsible for the EBI, an organizational executive, referral sources, and other health professionals (e.g., nurses, allied health). Between five and ten interviews were conducted at each site. Interview times ranged from 16 to 72 min, most lasting around 40 min per participant.

Document review

A checklist informed by case study literature was developed outlining the typical documents the research team was seeking [ 18 ]. The types of documents sought to review included job descriptions, strategic plans/planning documents, operating procedures and organizational policies, communications (e.g., website, media releases, email, meeting minutes), annual reports, administrative databases/files, evaluation reports, third party consultant reports, and routinely collected numerical data that measured implementation outcomes [ 27 ]. As each document was identified, it was numbered, dated, and recorded in the case study database with a short description of the content related to the research aims and the corresponding IRLM construct. Between 24 and 33 documents were accessed at each site. A total of 116 documents were reviewed across the case sites.

Naturalistic observations

The onsite observations occurred over 1 week, wherein typical organizational operations were viewed. The research team interacted with staff, asked questions, and sought clarification of what was being observed; however, they did not disrupt the usual work routines. Observations allowed us to understand how the exercise EBI operated and contrast that with documented processes and procedures. They also provided the opportunity to observe non-verbal cues and interactions between staff. While onsite, case notes were recorded directly into the case study database [ 62 , 63 ]. Between 15 and 40 h were spent on observations per site. A total of 95 h was spent across sites on direct observations.

Program sustainability assessment tool (survey)

The PSAT is a planning and evaluation tool that assesses the sustainability of an intervention across eight domains [ 64 , 65 , 66 ]: (1) environmental support, (2) funding stability, (3) partnerships, (4) organizational capacity, (5) program evaluation, (6) program adaption, (7) communication, and (8) strategic planning [ 64 , 65 ]. The PSAT was administered to a subset of at least three participants per site who completed the semi-structured interview. The results were then pooled to provide an organization-wide view of EBI sustainability. Three participants per case site are consistent with previous studies that have used the tool [ 67 , 68 ] and recommendations for appropriate use [ 65 , 69 ].

We included a validated measure of sustainability, recognizing calls to improve understanding of this aspect of implementation [ 70 , 71 , 72 ]. Noting the limited number of measurement tools for evaluating sustainability [ 73 ], the PSAT’s characteristics displayed the best alignment with the study aims. To determine “best alignment,” we deferred to a study by Lennox and colleagues that helps researchers select suitable measurement tools based on the conceptualization of sustainability in the study [ 71 ]. The PSAT provides a multi-level view of sustainability. It is a measurement tool that can be triangulated with other implementation frameworks, such as the CFIR [ 74 ], to interrogate better and understand the later stages of implementation. Further, the tool provides a contemporary account of an EBIs capacity for sustainability [ 75 ]. This is consistent with case study methods, which explore complex, contemporary, real-life phenomena.

The voluminous data collection that is possible through case studies, and is often viewed as a challenge of the method [ 19 ], was advantageous to developing the IRLM in the exemplar and identifying the causal pathways. First, it aided three types of triangulation through the study (method, theory, and data source triangulation) [ 76 ]. Method triangulation involved collecting evidence via four methods: interview, observations, document review, and survey. Theoretical triangulation involved applying two frameworks and one taxonomy to understand and interpret the findings. Data source triangulation involved selecting participants with different roles within the organization to gain multiple perspectives about the phenomena being studied. Second, data collection facilitated depth and nuance in detailing determinants and implementation strategies. For the determinant analysis, this illuminated the subtleties within context and improved confidence and accuracy for prioritizing determinants. As case studies are essentially “naturalistic” studies, they provide insight into strategies that are implementable in pragmatic settings. Finally, the design’s flexibility enabled the integration of a survey and routinely collected numerical data as evaluation measures for implementation outcomes. This allowed us to contrast “numbers” against participants’ subjective experience of implementation [ 77 ].

Data analysis

Descriptive statistics were calculated for the PSAT and combined with the three other data sources wherein framework analysis [ 78 , 79 ] was used to analyze the data. Framework analysis includes five main phases: familiarization, identifying a thematic framework, indexing, charting, and mapping and interpretation [ 78 ]. Familiarization occurred concurrently with data collection, and the thematic frame was aligned to the two frameworks and one taxonomy we applied to the IRLM. To index and chart the data, the raw data was uploaded into NVivo 12 [ 80 ]. Codes were established to guide indexing that aligned with the thematic frame. That is, determinants within the CFIR [ 47 ], implementation strategies listed in ERIC [ 48 ], and the implementation outcomes [ 49 ] of acceptability, fidelity, penetration, and sustainability were used as codes in NVivo 12. This process produced a framework matrix that summarized the information housed under each code at each case site.

The final step of framework analysis involves mapping and interpreting the data. We used the IRLM to map and interpret the data in the exemplar. First, we identified the core elements of the implemented exercise EBI. Next, we applied the CFIR valance and strength coding to prioritize the contextual determinants. Then, we identified the implementation strategies used to address the contextual determinants. Finally, we provided a rationale (a causal mechanism) for how these strategies worked to address barriers and contribute to specific implementation outcomes. The systematic approach advocated by the IRLM provided a transparent representation of the causal pathway underpinning the implementation of the exercise EBIs. This process was followed at each case site to produce an IRLM for each organization. To compare, contrast, and synthesize findings across sites, we identified the similarities and differences in the individual IRLMs and then developed an IRLM that explained a generalized process for implementation. Through the development of the causal pathway and mechanisms, we deferred to existing literature seeking to establish these relationships [ 81 , 82 , 83 ]. Aligned with case study methods, this facilitated an iterative process of constant comparison and challenging the proposed causal relationships. Smith and colleagues advise that the IRLM “might be viewed as a somewhat simplified format,” and users are encouraged to “iterate on the design of the IRLM to increase its utility” [ 7 ]. Thus, we re-designed the IRLM within a traditional logic model structure to help make sense of the data collected through the case studies. Figure 1 depicts the conceptual frame for the study and provides a graphical representation of how the IRLM pathway was produced.

The results are presented with reference to the three principles of the IRLM: comprehensiveness, indicating the key conceptual relationship and specifying critical study design . The case study method allowed for comprehensiveness through the data collection and analysis described above. The mean number of data sources informing the analysis and development of the causal pathway at each case site was 63.75 (interviews ( M = 7), observational hours ( M =23.75), PSAT ( M =4), and document review ( M = 29). This resulted in more than 30 determinants and a similar number of implementation strategies identified at each site (determinant range per site = 33–44; implementation strategy range per site = 36–44). Developing a framework matrix meant that each determinant (prioritized and other), implementation strategy, and implementation outcome were captured. The matrix provided a direct link to the data sources that informed the content within each construct. An example from each construct was collated alongside the summary to evidence the findings.

The key conceptual relationship was articulated in a traditional linear process by aligning determinant → implementation strategy → mechanism → implementation outcome, as per the IRLM. To synthesize findings across sites, we compared and contrasted the results within each of the individual IRLM and extracted similar elements to develop a generalized IRLM that represents cross-case findings. By redeveloping the IRLM within a traditional logic model structure, we added visual representations of the bi-directional and dependent relationships, illuminating the dynamism within the implementation process. To illustrate, intervention adaptability was a prioritized determinant and enabler across sites. Healthcare providers recognized that adapting and tailoring exercise EBIs increased “fit” with consumer needs. This also extended to adapting how healthcare providers referred consumers to exercise so that it was easy in the context of their other work priorities. Successful adaption was contingent upon a qualified workforce with the required skills and competencies to enact change. Different implementation strategies were used to make adaptions across sites, such as promoting adaptability and using data experts. However, despite the different strategies, successful adaptation created positive bi-directional relationships. That is, healthcare providers’ confidence and trust in the EBI grew as consumer engagement increased and clinical improvements were observed. This triggered greater engagement with the EBI (e.g., acceptability → penetration → sustainability), albeit the degree of engagement differed across sites. Figure 2 illustrates this relationship within the IRLM and provides a contrasting relationship by highlighting how a prioritized barrier across sites (available resources) was addressed.

Example of intervention adaptability (E) contrasted with available resources (B) within a synthesised IRLM across case sites

The final principle is to specify critical study design , wherein we have described how case study methodology was used to develop the IRLM exemplar. Our intention was to produce an explanatory causal pathway for the implementation process. The implementation outcomes of acceptability and fidelity were measured at the level of the provider, and penetration and sustainability were measured at the organizational level [ 49 ]. Service level and clinical level outcomes were not identified for a priori measurement throughout the study. We did identify evidence of clinical outcomes that supported our overall findings via the document review. Historical evaluations on the service indicated patients increased their exercise level or demonstrated a change in symptomology/function. The implementation strategies specified in the study were those chosen by the organizations. We did not attempt to augment routine practice or change implementation outcomes by introducing new strategies. The barriers across sites were represented with a (B) symbol and enablers with an (E) symbol in the IRLM. In the individual IRLM, consistent determinants and strategies were highlighted (via bolding) to support extraction. Finally, within the generalized IRLM, the implementation strategies are grouped according to the ERIC taxonomy category. This accounts for the different strategies applied to achieve similar outcomes across case studies.

This study provides a comprehensive overview that uses case study methodology to develop an IRLM in an implementation research project. Using an exemplar that examines implementation in different healthcare settings, we illustrate how the IRLM (that documents the causal pathways and mechanisms) was developed and enabled the synthesis of findings across sites.

Case study methodologies are fraught with inconsistencies in terminology and approach. We adopted the method described by Yin. Its guiding paradigm, which is rooted in objectivity, means it can be viewed as less flexible than other approaches [ 46 , 84 ]. We found the approach offered sufficient flexibility within the frame of a defined process. We argue that the defined process adds to the rigor and reproducibility of the study, which is consistent with the principles of implementation science. That is, accessing multiple sources of evidence, applying replication logic to select cases, maintaining a case study database, and developing logic models to establish causal pathways, demonstrates the reliability and validity of the study. The method was flexible enough to embed the IRLM within multiple phases of the study design, including conceptualization, philosophical alignment, and analysis. Paparini and colleagues [ 85 ] are developing guidance that recognizes the challenges and unmet value of case study methods for implementation research. This work, supported by the UK Medical Research Council, aims to enhance the conceptualization, application, analysis, and reporting of case studies. This should encourage and support researchers to use case study methods in implementation research with increased confidence.

The IRLM produced a relatively linear depiction of the relationship between context, strategies, and outcomes in our exemplar. However, as noted by the authors of the IRLM, the implementation process is rarely linear. If the tool is applied too rigidly, it may inadvertently depict an overly simplistic view of a complex process. To address this, we redeveloped the IRLM within a traditional logic model structure, adding visual representations of the dependent and bidirectional relationships evident within the general IRLM pathway [ 86 ]. Further, developing a general IRLM of cross-case findings that synthesized results involved a more inductive approach to identifying and extracting similar elements. It required the research team to consider broader patterns in the data before offering a prospective account of the implementation process. This was in contrast to the earlier analysis phases that directly mapped determinants and strategies to the CFIR and ERIC taxonomy. We argue that extracting similar elements is analogous to approaches that have variously been described as portable elements [ 87 ], common elements [ 88 ], or generalization by mechanism [ 89 ]. While defined and approached slightly differently, these approaches aim to identify elements frequently shared across effective EBIs and thus can form the basis of future EBIs to increase their utility, efficiency, and effectiveness [ 88 ]. We identified similarities related to determinant detail and mechanism of different implementation strategies across sites. This finding supports the view that many implementation strategies could be suitable, and selecting the “right mix” is challenging [ 16 ]. Identifying common mechanisms, such as increased motivation, skill acquisition, or optimizing workflow, enabled elucidation of the important functions of strategies. This can help inform the selection of appropriate strategies in future implementation efforts.

Finally, by developing individual IRLMs and then re-producing a general IRLM, we synthesized findings across sites and offered generalized findings. The ability to generalize from case studies is debated [ 89 , 90 ], with some considering the concept a fallacy [ 91 ]. That is, the purpose of qualitative research is to develop a richness through data that is situated within a unique context. Trying to extrapolate from findings is at odds with exploring unique context. We suggest the method described herein and the application of IRLM could be best applied to a form of generalization called ‘transferability’ [ 91 , 92 ]. This suggests that findings from one study can be transferred to another setting or population group. In this approach, the new site takes the information supplied and determines those aspects that would fit with their unique environment. We argue that elucidating the implementation process across multiple sites improves the confidence with which certain “elements” could be applied to future implementation efforts. For example, our approach may also be helpful for multi-site implementation studies that use methods other than case studies. Developing a general IRLM through study conceptualization could identify consistencies in baseline implementation status across sites. Multi-site implementation projects may seek to introduce and empirically test implementation strategies, such as via a cluster randomized controlled trial [ 93 ]. Within this study design, baseline comparison between control and intervention sites might extend to a comparison of organizational type, location and size, and individual characteristics, but not the chosen implementation strategies [ 94 ]. Applying the approach described within our study could enhance our understanding of how to support effective implementation.

Limitations

After the research team conceived this study, the authors of the PSAT validated another tool for use in clinical settings (Clinical Sustainability Assessment Tool (CSAT)) [ 95 ]. This tool appears to align better with our study design due to its explicit focus on maintaining structured clinical care practices. The use of multiple data sources and consistency in some elements across the PSAT and CSAT should minimize the limitations in using the PSAT survey tool. At most case sites, limited staff were involved in developing and implementing exercise EBI. Participants who self-selected for interviews may be more invested in assuring positive outcomes for the exercise EBI. Inviting participants from various roles was intended to reduce selection bias. Finally, we recognize recent correspondence suggesting the IRLM misses a critical step in the causal pathway. That is the mechanism between determinant and selection of an appropriate implementation strategy [ 96 ]. Similarly, Lewis and colleagues note that additional elements, including pre-conditions, moderators, and mediators (distal and proximal), exist within the causal pathway [ 13 ]. Through the iterative process of developing the IRLM, decisions were made about the determinant → implementation strategy relationship; however, this is not captured in the IRLM. Secondary analysis of the case study data would allow elucidation of these relationships, as this information can be extracted through the case study database. This was outside the scope of the exemplar study.

Developing an IRLM via case study methods proved useful in identifying causal pathways and mechanisms. The IRLM can complement and enhance the study design by providing a consistent and structured approach. In detailing our approach, we offer an example of how multiple case study designs that embed the IRLM can aid the synthesis of findings across sites. It also provides a method that can be replicated in future studies. Such transparency adds to the quality, reliability, and validity of implementation research.

Availability of data and materials

The data that support the findings of this study are available on request from the corresponding author [LC]. The data are not publicly available due to them containing information that could compromise research participant privacy.

Presseau J, McCleary N, Lorencatto F, Patey AM, Grimshaw JM, Francis JJ. Action, actor, context, target, time (AACTT): a framework for specifying behaviour. Implement Sci. 2019;14(1):102.

Article   PubMed   PubMed Central   Google Scholar  

Damschroder LJ. Clarity out of chaos: use of theory in implementation research. Psychiatry Res. 2020;283(112461).

Bauer M, Damschroder L, Hagedorn H, Smith J, Kilbourne A. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3(1):32.

Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10(1):53.

Lynch EA, Mudge A, Knowles S, Kitson AL, Hunter SC, Harvey G. “There is nothing so practical as a good theory”: a pragmatic guide for selecting theoretical approaches for implementation projects. BMC Health Serv Res. 2018;18(1):857.

Birken SA, Powell BJ, Presseau J, Kirk MA, Lorencatto F, Gould NJ, et al. Combined use of the Consolidated Framework for Implementation Research (CFIR) and the Theoretical Domains Framework (TDF): a systematic review. Implement Sci. 2017;12(1):2.

Smith JD, Li DH, Rafferty MR. The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. 2020;15(1):84.

Kellogg WK. Foundation. Logic model development guide. Michigan, USA; 2004.

McLaughlin JA, Jordan GB. Logic models: a tool for telling your programs performance story. Eval Prog Plann. 1999;22(1):65–72.

Article   Google Scholar  

Anselmi L, Binyaruka P, Borghi J. Understanding causal pathways within health systems policy evaluation through mediation analysis: an application to payment for performance (P4P) in Tanzania. Implement Sci. 2017;12(1):10.

Lewis C, Boyd M, Walsh-Bailey C, Lyon A, Beidas R, Mittman B, et al. A systematic review of empirical studies examining mechanisms of implementation in health. Implement Sci. 2020;15(1):21.

Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. 2019;7(3).

Lewis CC, Klasnja P, Powell BJ, Lyon AR, Tuzzio L, Jones S, Walsh-Bailey C and Weiner B. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Health. 2018;6(136).

Bartholomew L, Parcel G, Kok G. Intervention mapping: a process for developing theory and evidence-based health education programs. Health Educ Behav. 1998;25(5):545–63.

Article   CAS   PubMed   Google Scholar  

Weiner BJ, Lewis MA, Clauser SB, Stitzenberg KB. In search of synergy: strategies for combining interventions at multiple levels. JNCI Monographs. 2012;2012(44):34–41.

Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen J, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44(2):177–94.

Fernandez ME, ten Hoor GA, van Lieshout S, Rodriguez SA, Beidas RS, Parcel G, Ruiter R, Markham C and Kok G. Implementation mapping: using intervention mapping to develop implementation strategies. Frontiers. Public Health. 2019;7(158).

Yin R. Case study research and applications design and methods. 6th Edition ed. United States of America: Sage Publications; 2018.

Google Scholar  

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Med Res Methodol. 2011;11:100.

Stake R. The art of case study reseach. United States of America: Sage Publications; 2005.

Thomas G. How to do your case study. 2nd Edition ed. London: Sage Publications; 2016.

Ramanadhan S, Revette AC, Lee RM and Aveling E. Pragmatic approaches to analyzing qualitative data for implementation science: an introduction. Implement Sci Commun. 2021;2(70).

National Cancer Institute. Qualitative methods in implementation science United States of America: National Institutes of Health Services; 2018.

Mathers J, Taylor R, Parry J. The challenge of implementing peer-led interventions in a professionalized health service: a case study of the national health trainers service in England. Milbank Q. 2014;92(4):725–53.

Powell BJ, Proctor EK, Glisson CA, Kohl PL, Raghavan R, Brownson RC, et al. A mixed methods multiple case study of implementation as usual in children’s social service organizations: study protocol. Implement Sci. 2013;8(1):92.

van de Glind IM, Heinen MM, Evers AW, Wensing M, van Achterberg T. Factors influencing the implementation of a lifestyle counseling program in patients with venous leg ulcers: a multiple case study. Implement Sci. 2012;7(1):104.

Greenhalgh T, Macfarlane F, Barton-Sweeney C, Woodard F. “If we build it, will it stay?” A case study of the sustainability of whole-system change in London. Milbank Q. 2012;90(3):516–47.

Urquhart R, Kendell C, Geldenhuys L, Ross A, Rajaraman M, Folkes A, et al. The role of scientific evidence in decisions to adopt complex innovations in cancer care settings: a multiple case study in Nova Scotia, Canada. Implement Sci. 2019;14(1):14.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Herinckx H, Kerlinger A, Cellarius K. Statewide implementation of high-fidelity recovery-oriented ACT: A case study. Implement Res Pract. 2021;2:2633489521994938.

Young AM, Hickman I, Campbell K, Wilkinson SA. Implementation science for dietitians: The ‘what, why and how’ using multiple case studies. Nutr Diet. 2021;78(3):276–85.

Article   PubMed   Google Scholar  

van Zelm R, Coeckelberghs E, Sermeus W, Wolthuis A, Bruyneel L, Panella M, et al. A mixed methods multiple case study to evaluate the implementation of a care pathway for colorectal cancer surgery using extended normalization process theory. BMC Health Serv Res. 2021;21(1):11.

Albers B, Shlonsky A, Mildon R. Implementation Science 3.0. Switzerland: Springer; 2020.

Book   Google Scholar  

Hollick RJ, Black AJ, Reid DM, McKee L. Shaping innovation and coordination of healthcare delivery across boundaries and borders. J Health Organ Manag. 2019;33(7/8):849–68.

Article   PubMed Central   Google Scholar  

Pedersen B, Saltin B. Exercise as medicine – evidence for prescribing exercise as therapy in 26 different chronic diseases. Scand J Med Sci Sports. 2015;25:1–72.

Firth J, Siddiqi N, Koyanagi A, Siskind D, Rosenbaum S, Galletly C, et al. The Lancet Psychiatry Commission: a blueprint for protecting physical health in people with mental illness. Lancet Psychiatry. 2019;6(8):675–712.

Campbell K, Winters-Stone K, Wisekemann J, May A, Schwartz A, Courneya K, et al. Exercise guidelines for cancer survivors: consensus statement from international multidisciplinary roundtable. Med Sci Sports Exerc. 2019;51(11):2375–90.

Deenik J, Czosnek L, Teasdale SB, Stubbs B, Firth J, Schuch FB, et al. From impact factors to real impact: translating evidence on lifestyle interventions into routine mental health care. Transl Behav Med. 2020;10(4):1070–3.

Suetani S, Rosenbaum S, Scott JG, Curtis J, Ward PB. Bridging the gap: What have we done and what more can we do to reduce the burden of avoidable death in people with psychotic illness? Epidemiol Psychiatric Sci. 2016;25(3):205–10.

Article   CAS   Google Scholar  

Stanton R, Rosenbaum S, Kalucy M, Reaburn P, Happell B. A call to action: exercise as treatment for patients with mental illness. Aust J Primary Health. 2015;21(2):120–5.

Rosenbaum S, Hobson-Powell A, Davison K, Stanton R, Craft LL, Duncan M, et al. The role of sport, exercise, and physical activity in closing the life expectancy gap for people with mental illness: an international consensus statement by Exercise and Sports Science Australia, American College of Sports Medicine, British Association of Sport and Exercise Science, and Sport and Exercise Science New Zealand. Transll J Am Coll Sports Med. 2018;3(10):72–3.

Chambers D, Vinson C, Norton W. Advancing the science of implementation across the cancer continuum. United States of America: Oxford University Press Inc; 2018.

Schmitz K, Campbell A, Stuiver M, Pinto B, Schwartz A, Morris G, et al. Exercise is medicine in oncology: engaging clinicians to help patients move through cancer. Cancer J Clin. 2019;69(6):468–84.

Santa Mina D, Alibhai S, Matthew A, Guglietti C, Steele J, Trachtenberg J, et al. Exercise in clinical cancer care: a call to action and program development description. Curr Oncol. 2012;19(3):9.

Czosnek L, Rankin N, Zopf E, Richards J, Rosenbaum S, Cormie P. Implementing exercise in healthcare settings: the potential of implementation science. Sports Med. 2020;50(1):1–14.

Harrison H, Birks M, Franklin R, Mills J. Case study research: foundations and methodological orientations. Forum: Qualitative. Soc Res. 2017;18(1).

Yazan B. Three approaches to case study methods in education: Yin, Merriam, and Stake. Qual Rep. 2015;20(2):134–52.

Damschroder L, Aaron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21.

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Pol Ment Health. 2011;38(2):65–76.

Heale R, Twycross A. What is a case study? Evid Based Nurs. 2018;21(1):7–8.

Brownson R, Colditz G, Proctor E. Dissemination and implementation research in health: translating science to practice. Second ed. New York: Oxford University Press; 2017.

Quiñones MM, Lombard-Newell J, Sharp D, Way V, Cross W. Case study of an adaptation and implementation of a Diabetes Prevention Program for individuals with serious mental illness. Transl Behav Med. 2018;8(2):195–203.

Wiltsey Stirman S, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. 2019;14(1):58.

Baxter P, Jack S. Qualitative case study methodology: study design and implementation for novice researchers. Qual Rep. 2008;13(4):544–59.

Consolidated Framework for Implementation Research 2018. Available from: http://www.cfirguide.org/index.html . Cited 2018 14 February.

Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Admin Pol Ment Health. 2015;42(5):533–44.

Francis JJ, Johnston M, Robertson C, Glidewell L, Entwistle V, Eccles MP, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health. 2010;25(10):1229–45.

Teddlie C, Yu F. Mixed methods sampling: a typology with examples. J Mixed Methods Res. 2007;1(1):77–100.

Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. 2018;52(4):1893–907.

Braun V, Clarke V. To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qual Res Sport Exerc Health. 2021;13(2):201–16.

Burau V, Carstensen K, Fredens M, Kousgaard MB. Exploring drivers and challenges in implementation of health promotion in community mental health services: a qualitative multi-site case study using Normalization Process Theory. BMC Health Serv Res. 2018;18(1):36.

Phillippi J, Lauderdale J. A guide to field notes for qualitative research: context and conversation. Qual Health Res. 2018;28(3):381–8.

Mulhall A. In the field: notes on observation in qualitative research. J Adv Nurs. 2003;41(3):306–13.

Schell SF, Luke DA, Schooley MW, Elliott MB, Herbers SH, Mueller NB, et al. Public health program capacity for sustainability: a new framework. Implement Sci. 2013;8(1):15.

Washington University. The Program Sustainability Assessment Tool St Louis: Washington University; 2018. Available from: https://sustaintool.org/ . Cited 2018 14 February.

Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The Program Sustainability Assessment Tool: a new instrument for public health programs. Prev Chronic Dis. 2014;11:E12.

Stoll S, Janevic M, Lara M, Ramos-Valencia G, Stephens TB, Persky V, et al. A mixed-method application of the Program Sustainability Assessment Tool to evaluate the sustainability of 4 pediatric asthma care coordination programs. Prev Chronic Dis. 2015;12:E214.

Kelly C, Scharff D, LaRose J, Dougherty NL, Hessel AS, Brownson RC. A tool for rating chronic disease prevention and public health interventions. Prev Chronic Dis. 2013;10:E206.

Calhoun A, Mainor A, Moreland-Russell S, Maier RC, Brossart L, Luke DA. Using the Program Sustainability Assessment Tool to assess and plan for sustainability. Prev Chronic Dis. 2014;11:E11.

Proctor E, Luke D, Calhoun A, McMillen C, Brownson R, McCrary S, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10(1):88.

Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare. Implement Sci. 2018;13(1):27.

Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implement Sci. 2017;12(1):110.

Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015;10(1):155.

Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to enhance sustainability: addressing dynamic context and promoting health equity over time. Front Public Health. 2020;8(134).

Moullin JC, Sklar M, Green A, Dickson KS, Stadnick NA, Reeder K, et al. Advancing the pragmatic measurement of sustainment: a narrative review of measures. Implement Sci Commun. 2020;1(1):76.

Denzin N. The research act: A theoretical introduction to sociological methods. New Jersey: Transaction Publishers; 1970.

Grant BM, Giddings LS. Making sense of methodologies: a paradigm framework for the novice researcher. Contemp Nurse. 2002;13(1):10–28.

Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13(1):117.

Pope C, Ziebland S, Mays N. Qualitative research in health care. Analysing qualitative data. BMJ. 2000;320(7227):114–6.

QSR International. NVivo 11 Pro for Windows 2018. Available from: https://www.qsrinternational.com/nvivo-qualitative-data-analysissoftware/home .

Waltz TJ, Powell BJ, Fernández ME, Abadie B, Damschroder LJ. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implement Sci. 2019;14(1):42.

Michie S, Johnston M, Rothman AJ, de Bruin M, Kelly MP, Carey RN, et al. Developing an evidence-based online method of linking behaviour change techniques and theoretical mechanisms of action: a multiple methods study. Southampton (UK): NIHR Journals. Library. 2021;9:1.

Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26–33.

Ebneyamini S, Sadeghi Moghadam MR. Toward developing a framework for conducting case study research. Int J Qual Methods. 2018;17(1):1609406918817954.

Paparini S, Green J, Papoutsi C, Murdoch J, Petticrew M, Greenhalgh T, et al. Case study research for better evaluations of complex interventions: rationale and challenges. BMC Med. 2020;18(1):301.

Sarkies M, Long JC, Pomare C, Wu W, Clay-Williams R, Nguyen HM, et al. Avoiding unnecessary hospitalisation for patients with chronic conditions: a systematic review of implementation determinants for hospital avoidance programmes. Implement Sci. 2020;15(1):91.

Koorts H, Cassar S, Salmon J, Lawrence M, Salmon P, Dorling H. Mechanisms of scaling up: combining a realist perspective and systems analysis to understand successfully scaled interventions. Int J Behav Nutr Phys Act. 2021;18(1):42.

Engell T, Kirkøen B, Hammerstrøm KT, Kornør H, Ludvigsen KH, Hagen KA. Common elements of practice, process and implementation in out-of-school-time academic interventions for at-risk children: a systematic review. Prev Sci. 2020;21(4):545–56.

Bengtsson B, Hertting N. Generalization by mechanism: thin rationality and ideal-type analysis in case study research. Philos Soc Sci. 2014;44(6):707–32.

Tsang EWK. Generalizing from research findings: the merits of case studies. Int J Manag Rev. 2014;16(4):369–83.

Polit DF, Beck CT. Generalization in quantitative and qualitative research: myths and strategies. Int J Nurs Stud. 2010;47(11):1451–8.

Adler C, Hirsch Hadorn G, Breu T, Wiesmann U, Pohl C. Conceptualizing the transfer of knowledge across cases in transdisciplinary research. Sustain Sci. 2018;13(1):179–90.

Wolfenden L, Foy R, Presseau J, Grimshaw JM, Ivers NM, Powell BJ, et al. Designing and undertaking randomised implementation trials: guide for researchers. BMJ. 2021;372:m3721.

Nathan N, Hall A, McCarthy N, Sutherland R, Wiggers J, Bauman AE, et al. Multi-strategy intervention increases school implementation and maintenance of a mandatory physical activity policy: outcomes of a cluster randomised controlled trial. Br J Sports Med. 2022;56(7):385–93.

Malone S, Prewitt K, Hackett R, Lin JC, McKay V, Walsh-Bailey C, et al. The Clinical Sustainability Assessment Tool: measuring organizational capacity to promote sustainability in healthcare. Implement Sci Commun. 2021;2(1):77.

Sales AE, Barnaby DP, Rentes VC. Letter to the editor on “the implementation research logic model: a method for planning, executing, reporting, and synthesizing implementation projects” (Smith JD, Li DH, Rafferty MR. the implementation research logic model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. 2020;15 (1):84. Doi:10.1186/s13012-020-01041-8). Implement Sci. 2021;16(1):97.

Download references

Acknowledgements

The authors would like to acknowledge the healthcare organizations and staff who supported the study.

SR is funded by an NHMRC Early Career Fellowship (APP1123336). The funding body had no role in the study design, data collection, data analysis, interpretation, or manuscript development.

Author information

Authors and affiliations.

Mary MacKillop Institute for Health Research, Australian Catholic University, Melbourne, Australia

Louise Czosnek & Eva M. Zopf

Cabrini Cancer Institute, The Szalmuk Family Department of Medical Oncology, Cabrini Health, Melbourne, Australia

Eva M. Zopf

Peter MacCallum Cancer Centre, Melbourne, Australia

Prue Cormie

Sir Peter MacCallum Department of Oncology, The University of Melbourne, Melbourne, Australia

Discipline of Psychiatry and Mental Health, University of New South Wales, Sydney, Australia

Simon Rosenbaum

School of Health Sciences, University of New South Wales, Sydney, Australia

Faculty of Health, Victoria University of Wellington, Wellington, New Zealand

Justin Richards

Faculty of Medicine and Health, University of Sydney, Sydney, Australia

Nicole M. Rankin

Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, Australia

You can also search for this author in PubMed   Google Scholar

Contributions

LC, EZ, SR, JR, PC, and NR contributed to the conceptualization of the study. LC undertook the data collection, and LC, EZ, SR, JR, PC, and NR supported the analysis. The first draft of the manuscript was written by LC with NR and EZ providing first review. LC, EZ, SR, JR, PC, and NR commented on previous versions of the manuscript and provided critical review. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Louise Czosnek .

Ethics declarations

Ethics approval and consent to participate.

This study is approved by Sydney Local Health District Human Research Ethics Committee - Concord Repatriation General Hospital (2019/ETH11806). Ethical approval is also supplied by Australian Catholic University (2018-279E), Peter MacCallum Cancer Centre (19/175), North Sydney Local Health District - Macquarie Hospital (2019/STE14595), and Alfred Health (516-19).

Consent for publication

Not applicable.

Competing interests

PC is the recipient of a Victorian Government Mid-Career Research Fellowship through the Victorian Cancer Agency. PC is the Founder and Director of EX-MED Cancer Ltd, a not-for-profit organization that provides exercise medicine services to people with cancer. PC is the Director of Exercise Oncology EDU Pty Ltd, a company that provides fee for service training courses to upskill exercise professionals in delivering exercise to people with cancer.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Standards for Reporting Qualitative Research (SRQR).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Czosnek, L., Zopf, E.M., Cormie, P. et al. Developing an implementation research logic model: using a multiple case study design to establish a worked exemplar. Implement Sci Commun 3 , 90 (2022). https://doi.org/10.1186/s43058-022-00337-8

Download citation

Received : 19 March 2022

Accepted : 01 August 2022

Published : 16 August 2022

DOI : https://doi.org/10.1186/s43058-022-00337-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Logic model
• Case study methods
• Causal pathways
• Causal mechanisms

Implementation Science Communications

ISSN: 2662-2211

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

No internet connection.

All search filters on the page have been cleared., your search has been saved..

• All content
• Dictionaries
• Encyclopedias
• Expert Insights
• Foundations
• How-to Guides
• Journal Articles
• Little Blue Books
• Little Green Books
• Project Planner
• Tools Directory
• Sign in to my profile My Profile

• Sign in Signed in
• My profile My Profile

Using Multiple Case Study Design and Thematic Analysis in NVivo to Analyze Value Co-Creation in Brand Alliances

• By: Ediz Edip Akçay
• Product: Sage Research Methods: Business
• Publisher: SAGE Publications Inc.
• Publication year: 2023
• Online pub date: March 21, 2023
• Discipline: Business and Management , Marketing
• Methods: Case study research , Thematic analysis , NVivo
• DOI: https:// doi. org/10.4135/9781529669305
• Keywords: alliances , branding , capabilities , competitive advantage , software Show all Show less
• Online ISBN: 9781529669305 Copyright: © 2023 SAGE Publications Ltd More information Less information

Multiple case study design is a qualitative approach that can be used to investigate a phenomenon in its real-world context. It allows the researcher to start with theory to develop research questions and explore them in different cases. While there are several approaches to analyzing findings in multiple case study designs, a six-phase approach to reflexive thematic analysis is a systematic way to evaluate the results. In this case study, readers will learn how to overcome the challenges related to data collection and analysis in a multiple case study design. The practical recommendations for choosing the participants will be discussed. Moreover, the application of reflexive thematic analysis in NVivo will be explored.

Learning Outcomes

By the end of this case study, readers should be able to:

• Explain how multiple case study design is used in qualitative analysis
• Employ reflexive thematic analysis for qualitative analysis using NVivo
• Assess the challenges in multiple case study design

Project Overview and Context

Research background.

Brand alliance is an increasingly popular business strategy to achieve a competitive advantage, which is defined as a marketing collaboration where two or more brands come together to form an alliance for a short or long period of time ( Yan & Cao, 2017 ). My professional experience in a company that facilitates brand alliances between companies from different product categories inspired me to develop my Ph.D. research around this topic. My previous research in the branding field helped me identify gaps related to value co-creation during brand alliances in a business-to-business (B2B) setting.

I needed to have a closer look at the topic to explore insights on how the value is co-created in brand alliances. This led to the choice of a qualitative research method, namely a multiple case study design. A multiple case study design enables the investigation of the selected phenomenon (i.e. cross-category brand alliance in this research context) in its natural setting ( Bonoma, 1985 ). A cross-category brand alliance occurs when brands from different product categories form a brand alliance ( Smarandescu et al., 2013 ). Considering the lack of research on the topic in an emerging market context, I used a convenience and purposive sampling approach to choose cases from my existing professional network in Turkey.

Research Aim and Objectives

There were several studies ( Decker & Baade, 2016 ; Smarandescu et al., 2013 ) that investigated brand alliances from the customer perspective using a quantitative approach to analyze how customers perceive the brands when they are in an alliance. However, the business side of brand alliances has been scarcely researched, making it difficult to understand how brands co-create value and what types of value they co-create. Therefore, my research aimed to investigate the marketing resources and capabilities in relation to alliance outcomes in cross-category brand alliances.

I investigated the value co-creation process during cross-category brand alliances to map the marketing resources and capabilities utilized by the partner brands. Furthermore, I identified the types of values co-created in these cross-category brand alliances. The unit of analysis in the research is the dyadic relationship between alliance partner brands. Underpinning my exploration of cross-category brand alliances were the resource-based view (RBV) and service-dominant (S-D) logic. RBV claims that the internal resources of a firm are the sources of the firm’s sustained competitive advantage. To achieve sustained competitive advantage, a firm should have valuable, rare, inimitable, and non-substitutable (VRIN) resources( Kraaijenbrink et al., 2010 ). S-D logic, introduced by Vargo and Lusch in 2004, suggests that marketing activity is better explained in terms of service-for-service exchange than in terms of goods-for-goods or goods-for-money exchange. Moreover, value is not created by one stakeholder but is co-created by the stakeholders ( Vargo & Lusch, 2017 ).

I collected data through a qualitative approach, conducting semi-structured interviews with the marketing managers of the brands in the chosen cases. I analyzed the data using thematic analysis in NVivo and proposed a theoretical framework to depict the relations between partner brands' resources and capabilities and cross-category brand alliance outcomes during the value co-creation process.

The aim of the research was to investigate which marketing resources and capabilities are in relation to alliance outcomes in cross-category brand alliances. The value co-creation process during the cross-category brand alliances was investigated to map the marketing resources and capabilities utilized by partner brands. Furthermore, the types of values co-created in the cross-category brand alliances were identified.

After analyzing the data using thematic analysis in NVivo, a theoretical framework was proposed to depict the relations of partner brands’ resources and capabilities with cross-category brand alliance outcomes during the value co-creation process. Given the increasing number of brand alliances in the recent years and my experience in the field, this topic was worth exploring for my Ph.D. research.

Section Summary

• Brand alliance is an increasingly popular business strategy to achieve a competitive advantage.
• There are rare studies that investigated the value co-creation process in brand alliances from a business-to-business perspective.
• The focus of the research project was investigating which marketing resources and capabilities are utilized in cross-category brand alliances.

Research Design

In the research, I aimed to explore the value co-creation process in brand alliances by understanding the experiences of the people involved in the process, which qualitative research enabled through an interpretive approach ( Jackson et al., 2007 ).

In business-to-business research, it is difficult to set up experimental studies, and studies are mostly based on simulated business organizations. However, case study research allows researchers to investigate the real-world businesses by analyzing the context and processes involved in the researched phenomenon ( Johnston et al., 1999 ). The case study method “explores a real-life, contemporary bounded system (a case) or multiple bounded systems (cases) over time, through detailed, in-depth data collection involving multiple sources of information… and reports a case description and case themes( Creswell & Poth, 2016, p. 97 ).” Therefore, I applied a case study approach to investigate brand alliances in their natural setting. To understand whether the findings were replicated in different cases, I used a multiple case study design as suggested by Yin (2011) . In a multiple case study design, the research includes more than one single case. The researcher investigates multiple cases to find out the differences and similarities between the cases. Furthermore, the analysis can be made within each case and across cases ( Gustafsson, 2017 ). Since I was investigating cross-category brand alliances, I aimed to choose cases from different product categories.

After deciding on the cases, I started to collect information about the cases from public sources such as company websites, industry reports, and social media accounts to have background information about the brand alliances they formed. However, this secondary data was not enough to enable deep insights into how value is co-created in the chosen cases. I decided to conduct semi-structured interviews not only with the marketing managers of the cases, but also with the marketing managers of the partner brands of the cases, who formed brand alliances with the case brands. On some occasions, there were also brand alliances between the cases, which provided additional insights about the value co-creation processes.

To develop the interview guide, I identified the main themes and subthemes after the literature review phase of the study. This is an important distinction between the ethnographic approach and the case study method: there is a need to consult theory in the case study method before collecting data ( Johnston et al., 1999 ). For each subtheme, I adapted questions from past studies to use in the interview guide. The interviews were planned as semi-structured to have the flexibility to ask additional questions. It was also critical to ensure all the questions were covered to achieve consistent analysis of cases.

While the interview guide was prepared in English, the native language of potential participants was another language, namely Turkish. Some of the cases were international companies that employed people with multilingual skills. However, language was still a potential barrier for participants who did not know English or who did not feel comfortable talking in a language other than their native one. Therefore, the questions were translated and back-translated by expert native speakers. I conducted a pilot interview to check the suitability and understandability of the questions. Some extra questions or words were removed and added to improve the interview guide after the pilot interview. I used the revised interview guide in further interviews with research participants. All the interviews were conducted in a face-to-face meeting environment and audio recorded with the consent of the participants. I also took notes about the visual cues of the participants during the interviews, which provided additional insights about the themes.

After the data collection, I first transcribed the interviews by listening to the audio recordings. After the transcription, the transcribed data, which were in the native language of the participants, were translated and back-translated by expert native speakers to ensure the meaning was preserved.

Once the translation and transcription of all interviews were completed, I had a vast amount of data to analyze. In addition to the interviews, I had secondary data about the cases and my observation notes during the interviews. I decided to analyze the data using the thematic analysis method and checked suitable tools to assist in this process. Thematic analysis is a useful qualitative research method for identifying, analyzing, and reporting patterns in data ( Braun & Clarke, 2006 ). Among the alternative qualitative data analysis tools, I chose NVivo considering the training opportunities I had during my Ph.D. degree. During the training, I focused on learning the steps of thematic analysis in NVivo. I used the pilot interview data to test and apply my learnings in NVivo, which gave me hands-on experience while learning how to use the tool.

The next step was choosing the thematic analysis approach to analyse the data. There are mainly three different approaches to thematic analysis which are reflexive (1) thematic analysis, (2) coding reliability thematic analysis, and (3) codebook thematic analysis. While coding reliability and codebook approaches adopt the use of a structured codebook, the reflexive approach is an accessible and flexible approach to qualitative data analysis which enables the researcher to identify themes in the collected data ( Byrne, 2022 ). After checking these different approaches to thematic analysis, I found that the six-phase reflexive thematic analysis approach developed by Braun and Clarke (2006) would be the most suitable to apply in NVivo. The six-phases they proposed are (1) familiarizing yourself with the data, (2) generating initial codes, (3) searching for themes, (4) reviewing themes, (5) defining and naming themes, and (6) producing the report. Although the names of the phases were slightly changed in their latest article ( Braun & Clarke, 2006 ), the authors established guidelines for reflexive thematic analysis which can be applied in NVivo.

By applying this approach, I first familiarized myself with the data during the transcription process. NVivo has different ways of coding such as drag-and-drop coding, right-click coding, and quick coding. It was helpful to use NVivo for coding to find commonalities and differences between the cases as the structured view presented the codes in a user-friendly dashboard. Still, it was my responsibility to generate the codes as I didn’t use the auto-coding option in NVivo. The initial codes helped me identify potential themes, which were later reviewed against the themes and subthemes of the interview guide. I created a different folder for new themes in NVivo to distinguish between the ones those informed by past studies and those that were constructed during the analysis. At this phase, it was important to develop the linkages between the themes and research questions. Later, I used the themes and subthemes to develop summary tables in the findings and discussion chapters of my Ph.D. thesis.

• The case study method is useful in a business-to-business context, as the researcher can investigate the real-world businesses by analyzing the context and processes involved in the researched phenomenon.
• Multiple case study design allows the researcher to understand if the findings are replicated in different settings.
• The six-phase approach to thematic analysis developed by Braun and Clarke (2006) is a systematic, reflexive thematic analysis method to analyze qualitative data in NVivo.

Research Practicalities

Because this was Ph.D. research, I had a limited time to complete the target number of interviews about the chosen cases. Although I intended to interview a total of 20 people in about five cases, this seemed impractical after I started contacting potential participants for interviews. The challenge was to arrange meetings with potential participants, who were busy marketing managers of the brands and who conducted brand alliances in the cases. An additional challenge was the changing roles and jobs of potential participants. Moreover, I was working full-time with a busy schedule at my workplace. I was doing a part-time Ph.D., which meant I needed to work on my Ph.D. research after my office hours or on weekends.

After realizing the potential challenges with the interview arrangements, I decided to start with my existing network before reaching out to other potential participants. Furthermore, I started to contact as many potential participants about the cases as I could to reach the target number of interviews. Eventually, I ended up with 11 interviews from 10 different brands for five cases, which is an acceptable number for a qualitative study according to IPA (Interpretative Phenomenological Analysis) guidelines ( Robinson, 2014 ).

During the interviews, it was critical to build rapport with the participant to make them feel comfortable before answering the questions. While I had previous encounters with some participants due to my professional experience, it was my first meeting with most of the participants for the interview. As they were devoting at least one hour of their busy schedule, they wanted to understand what the research was about and how it would be used. Moreover, they were sharing information about their company, which could be confidential or sensitive if shared with the public or with competitors.

To overcome these issues, I shared a participant information sheet with the potential participants before each interview to provide detailed information about the research, data protection and process measures, and their privacy. I also encouraged them to ask additional questions if anything was not clear before the interview. When they accepted to arrange an interview, I shared a participant agreement form with them to obtain their consent for an audio recording of the interview, and the use of their anonymized data for research purposes. They were also granted permission to withdraw from the research during or after the interview unless their anonymized data had already been used in the analysis.

Some participants were people I knew due to our professional relationships. When you are interviewing people you know, there are potential issues such as establishing trust and rapport, role conflict, and self-disclosure ( McConnell‐Henry et al., 2010 ). While having had a previous encounter sped up the process of building rapport, I needed to build the trust of the participants that I did not have any “hidden agenda” related to our professional roles. I clarified my role as a researcher at the start of the interview who aimed to collect data for Ph.D. research. Even though the confidentiality and anonymity of data were detailed in the participant agreement form, I explained the steps by which these were assured in the research, so as to establish the trust of the participants. The potential issues in interviewing people I knew were related not only to the information the participants shared, but also to the possibility of self-disclosure. To prevent over disclosure, I only asked questions based on the interview guide or based on the information shared by the participants during the interview.

Although the personal information about the participants was not used in the research, there was a need for further anonymization. The research was about real-life cross-category brand alliances, and it wouldn’t be difficult for someone living in Turkey to guess the companies mentioned in the research. To protect the identities of the research participants, their job roles and companies were anonymized in the study. As an ethical issue, anonymization should be considered carefully throughout the research process when there is a risk of sensitive, confidential information disclosure ( Clark, 2006 ). Letters from the Greek alphabet (e.g. α, β) were used to anonymize the company names. Instead of the company names, pseudonyms such as α1, β1 were used. The job roles of the participants were mentioned as a generalized roles for all participants (i.e. marketing manager of α1, marketing manager of β1). After the data analysis, any confidential or sensitive information was removed from the quotes that were used in the final version of my Ph.D. thesis.

As mentioned in the research design section, the interview guide was translated from English to Turkish by expert native speakers. Still, some participants struggled to understand the concepts in some of the questions. Thanks to my experience in the field, I provided further information about the meanings of the concepts to ensure the participants were providing the insights that the research aimed for. For translated interview guides that involve technical terms or concepts, I suggest that researchers have a note of definitions in the original language and the language of translation. The researcher can use the note of definitions to support the participant when they struggle to understand the words in the questions. Having a standard note of definitions for all interviews would also ensure consistency between the interviews.

On some occasions, some participants started to talk about a topic that was not part of the themes of the interview guide questions. Considering the participants’ time constraints, I managed to ask transition questions from the interview guide to bring the interview back on track without disturbing the flow of the interview. Although the topics they introduced were not always relevant to the themes, they were still helpful in enabling additional insights about the value co-creation processes in the brand alliances.

As the interviews were based on the previous brand alliances of the chosen cases, there were mentions of both positive and negative experiences that happened during the brand alliances. While the interviews were conducted separately on a one-to-one basis, it was important to preserve the confidentiality of the shared information. To ensure transparency of data collection, I needed to include one transcript when I submitted the final version of my Ph.D. thesis. After checking the transcripts, I chose the transcript that had a more neutral tone in the details to include in the appendix of the thesis.

When I started using NVivo as the data analysis tool for thematic analysis, I realized that it still takes a lot of time to code (creating nodes in NVivo 11) the data. In addition, I needed to ensure consistency between the transcripts that were originally in English and those that had been translated into English. To remove ambiguity about the order of analysis between the transcriptions, I contacted the NVivo instructor to get their ideas. They suggested starting with the English versions and moving to the translated versions while coding the data. This allowed me to correct any misunderstandings in the translation of transcripts.

• Arranging interviews with research participants in a business-to-business context can take a long time due to the busy schedules of the potential participants.
• It is not only the personal information but also the company information that should be anonymized to protect the identity of the participants.
• A pilot interview is useful to revise and finalize the interview guide, especially when the interview guide is translated into a different language.

Method in Action

Even with a structured and dedicated tool like NVivo, it took me around six months to complete the thematic analysis process, from the initial coding phase to the final report development phase. This was partly due to my personal time constraints, as I had a full-time job. Additionally, I cannot even imagine the amount of time I would have spent even on coding alone without the use of a software like NVivo. Still, anyone using NVivo should accept that there will still be a lot of time-consuming work involved in creating cases and codes.

Even though I had themes and subthemes in the interview guide that were informed by literature, I realized that the code categories (nodes in NVivo 11) I generated in NVivo did not completely match those in the interview guide. After reading more about the thematic analysis stages, I realized that I need to create codes at their latent level of meaning, rather than their semantic meaning (Braun and Clarke cited in Cooper et al., 2012 ).

At the initial coding phase of thematic analysis on NVivo, I created 20 main nodes, with several nodes connected to the main nodes. In total, 2500 pieces of content were coded among 115 thousand words at the initial coding phase. NVivo has a user-friendly dashboard that summarizes the number of connections to the interviews and sources. You can also easily see the coded pieces of content by double-clicking on the codes. This feature allows you to identify the relevant sections of the cases you have coded.

After evaluating the initial codes that were connected to the sources, I realized that some important codes were not connected to certain cases. I decided to revisit the interviews to understand why insights related to some codes were not mentioned. One of my objectives was to understand whether the findings were replicated in different cases. It wouldn’t be possible to reach this objective without the missing codes in some cases. That’s why I decided to ask additional questions to the participants of the cases which had missing insights. As it wasn’t easy to arrange new face-to-face meetings with the marketing managers, I shared the questions via email and explained the details through phone calls. After receiving the answers from the participants, I added them to the relevant cases in NVivo and coded them using the same approach as in the initial interviews.

When I had finished coding all the interviews with the supportive questions, I moved to the theme-searching phase. Braun and Clarke (2006, p.82) state that a theme “captures something important about the data in relation to the research question and represents some level of patterned response or meaning within the data set.” They also point out that themes don’t simply emerge; rather, the researcher is constructing or generating them while working with the data. Following this approach, I started to look for relations to my research questions, which aimed to identify the marketing resources and capabilities that had been utilized, and the types of values that had been co-created in the brand alliances. At the end of this phase, I generated a codebook in NVivo to identify which themes were related to which cases.

After the initial construction of themes, I moved to the fourth phase of the approach, which was reviewing themes. I shared the codebook with my supervisors to get their feedback on the initial themes. Based on their feedback, I revisited the studies which guided me in developing the themes and subthemes in the interview guide. This led me to the next phase, which was that of defining and naming themes. To ensure the themes were in line with the research questions and objectives, I realized I needed to revise the names of some themes while merging others.

At the final stage of the reflexive thematic analysis, which involved producing the report in my research, I began writing the findings and discussion chapters of my Ph.D. thesis. I used the output from the thematic analysis to name the headings and subheadings of my finding chapter, to develop summary tables related to themes. I used relevant quotes from the interviews by clicking on the theme and finding them on NVivo. Furthermore, I developed frameworks based on the thematic analysis to depict the relations of partner brands’ resources and capabilities with the cross-category brand alliance outcomes during the value co-creation process, which was one of the objectives of my Ph.D. research.

• Qualitative analysis tools such as NVivo help the researcher with a structured approach for coding and developing themes in thematic analysis.
• The themes do not emerge in the data, the researcher needs to construct the themes while working with the data.
• The researcher can use NVivo to construct themes in the data by analyzing the codes and by evaluating the relation of codes to the research questions.

Practical Lessons Learned

As I aimed to investigate cross-category brand alliances in my research, a multiple case study design was the most suitable approach to compare and contrast cases from different product categories. Moreover, analyzing multiple cases allowed me to validate insights that occurred in different categories. However, working with multiple cases increased the difficulty of finding interview participants for each case. To improve this aspect, I would recommend identifying as many cases as possible at the initial stage, instead of focusing on a limited number of cases. After that, the researcher can contact all the potential participants in the identified cases and narrow down the list to involve the cases in that the participants accept the interview request. This approach would shorten the time the researcher spends reaching the number of interviews needed for data collection.

While I started with a purposive sampling approach to choose the participants in the cases, it was a correct decision to include convenience sampling when I struggled to arrange meetings with the potential participants. Contacting the potential participants in my network sped up the process of arranging initial interviews. I suggest that researchers reach as many potential participants as they can before starting their data collection if they are planning to conduct interviews.

Even though I chose thematic analysis as the data analysis method, I struggled to find the correct approach to apply it. Many articles had vague explanations for generating codes and moving to the theme-searching stage. However, Braun and Clarke’s (2006) six-phase approach to reflexive thematic analysis provided a systematic approach to analyzing the data. Based on my experience applying this reflexive thematic analysis approach in multiple case study designs, I would suggest that researchers spend more time in the first phase, which is that of familiarizing yourself with the data. When you have familiarised yourself with the data you collected, making linkages between the cases becomes easier in the further phases of generating initial codes and searching for themes.

I was lucky to complete the interviews long before the pandemic, which gave me the opportunity to conduct them in a face-to-face environment. Having face-to-face meetings provided additional insights into the participants’ reactions to the questions, and their approach to answering them. In addition, it was easier to build rapport after meeting in a place where they were accustomed to having meetings. However, conducting interviews in an online environment such as Zoom could provide the opportunity to transcribe the interviews automatically. Later, I learned that there were tools such as Google Cloud’s speech-to-text and Transcribe software, which are dedicated audio-transcription tools. I would recommend trying audio-transcription tools to save time in transcribing the data if the researcher has a time constraint and has the budget to pay for their service. Still, the accuracy levels of such tools should be tested before making the decision to invest money and time in the tools. Moreover, transcribing the data yourself would contribute more to the “familiarising yourself with the data” phase of the thematic analysis.

Despite attending comprehensive training about NVivo, I didn’t have enough time to try and learn all the features of the software. My main focus was on learning NVivo to apply thematic analysis to the data I had started to collect. After starting to use NVivo to analyze the data, I watched many tutorial videos about NVivo’s other features. In one of them, I learned that NVivo could also be used in the literature review stage of the research by importing the articles to the software. It has features that allow you to highlight the sections of the articles and take notes about them, which can be useful for later reference. I found out that using NVivo for the literature review stage of research could potentially enhance it. Additionally, the notes on the articles would be useful for linking them to the codes and themes generated in the thematic analysis stage. NVivo also allows users to import audio files for analysis, which is another feature I would suggest researchers learn and try if they collect audio data for analysis.

Although NVivo software is dedicated to qualitative analysis, it requires the researcher to manually work on the data to generate codes and develop themes. While NVivo also has auto-coding features, it has some limitations. There are other automated qualitative analysis tools, such as Leximancer and ATLAS.ti, that researchers use for thematic analysis. Although I had the opportunity to receive training about NVivo, I didn’t have much time nor a sufficient budget to look for more alternatives while choosing an analysis software. However, I would suggest researchers to explore more qualitative analysis tools to analyze the data. Using a more sophisticated automated coding tool could provide additional insights to support the manual coding process for researchers.

• The researcher should prepare an extensive list of potential cases and participants in a multiple case study design before choosing the cases to ease the process of sampling.
• An audio-transcription tool can be helpful to shorten the time between data collection and analysis if the researcher has a time constraint
• Learning how to use the analysis software should start as soon as possible after the researcher decides 0n the analytical method.

Multiple case study design is an effective qualitative research approach for investigating a chosen phenomenon in its natural setting. It provides the opportunity to replicate the research in different contexts by collecting data through interviews, observations, focus groups, and secondary research. While qualitative research tends to involve subjective interpretations in its essence, thematic analysis can improve the rigor of the study in a multiple case study design. I benefited from using the six-phase approach to reflexive thematic analysis ( Braun & Clarke, 2006 ) to achieve the objectives of my Ph.D. research. This approach has systematic and structured phases which can be implemented using NVivo as the tool to support thematic analysis. The approach also enhanced the rigor and transparency of my methodology by setting the guidelines I followed in my data analysis.

For studies in a business-to-business context, thematic analysis using a multiple case study design would be especially useful. This type of analysis allows to a researcher to explore relationships and processes in the business environment within their specific contexts. Furthermore, the insights and recommendations developed as an output of the case study method would have the potential to influence marketing managers, for example when the case findings are used for training purposes. The case study evidence is seen as more persuasive than quantitative findings because of its vivid, personal, and concrete nature ( Johnston et al., 1999 ) However, a researcher who is investigating a business-to-consumer phenomenon might choose to use a quantitative approach to test the relationships between variables using a larger sample.

For a researcher who decides to apply thematic analysis, I would suggest using a supportive software such as NVivo, which eases the process of coding and developing themes. Although I did not use the auto-coding option in NVivo, the software helped me identify the missing pieces in the cases at the initial coding phase and develop the main themes and subthemes in the later phases of the thematic analysis. To summarise, thematic analysis using NVivo in a multiple casestudy designs is an efficient qualitative research method, especially for researchers who investigate a phenomenon in a business-to-business setting.

Classroom Discussion Questions

• 1. Why is a case study design more suitable to investigate a phenomenon in a business-to-business context?
• 2. How would you ensure the anonymity of the participants in qualitative research in a business-to-business context?
• 3. What would be the benefit of manual transcription of interviews in six-phase approach to thematic analysis?

Multiple-Choice Quiz Questions

1. Which is not a reason to anonymize the company names in the data analysis?

Incorrect Answer

Feedback: This is not the correct answer. The correct answer is B.

Correct Answer

Feedback: Well done, correct answer

2. Which one is a distinction between the case study method and ethnographic research?

Feedback: This is not the correct answer. The correct answer is C.

3. Which qualitative analysis software generates the topic models automatically?

Feedback: This is not the correct answer. The correct answer is A.

4. Which thematic analysis approach does NOT adopt the use of a structured codebook?

5. Which one is the first phase in Braun and Clarke’s six-phase approach to reflexive thematic analysis?

Further Reading

Web resources, sign in to access this content, get a 30 day free trial, more like this, sage recommends.

We found other relevant content for you on other Sage platforms.

Have you created a personal profile? Login or create a profile so that you can save clips, playlists and searches

• Sign in/register

Navigating away from this page will delete your results

Please save your results to "My Self-Assessments" in your profile before navigating away from this page.

Sign in to my profile

Sign up for a free trial and experience all Sage Learning Resources have to offer.

You must have a valid academic email address to sign up.

Get off-campus access

• View or download all content my institution has access to.

Sign up for a free trial and experience all Sage Research Methods has to offer.

• view my profile
• view my lists

Multiple case study design: the example of place marketing research

• Original Article
• Published: 05 February 2020
• Volume 17 , pages 50–62, ( 2021 )

Cite this article

• Marek Ćwiklicki   ORCID: orcid.org/0000-0002-5298-0210 1 &
• Kamila Pilch 1  

1310 Accesses

5 Citations

1 Altmetric

Explore all metrics

The objective of this paper is to discuss the specificity of multiple case study (MCS) research design using analysis of research description realized according to this strategy in the field of place branding and place marketing published between 1976 and 2016 in scholarly journals. Selecting cases and cases’ context are most frequently explained in place marketing articles where findings are results of MCS research. The choice of a case study as a research strategy and limitations of the studies are less frequently justified in investigated papers. Our analysis shows that the authors should pay more attention to elements characteristic for methodological rigour in their descriptions of the research method. For this purpose, we prepared a checklist. We have discussed in detail key methodological issues for MCS. Moreover, we have formulated guidelines for improving research methodology’s descriptions in scholarly papers. It should lead to an increase in methodological rigour in future research reports. Therefore, researchers will find out suggestions for studying phenomena within place branding domain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Place branding and place marketing: a contemporary analysis of the literature and usage of terminology

Navin Kumar & Rajeev Kumar Panda

Place branding research: a thematic review and future research agenda

Alok Acharya & Zillur Rahman

Stakeholders’ perceptions of place branding and the role of the public sector: An exploratory analysis

Maria J. Cerda-Bertomeu & Francisco J. Sarabia-Sanchez

Details of this study can be found in Vuignier’s studies ( 2016 , 2017 ).

Acharya, A., and Z. Rahman. 2016. Place branding research: A thematic review and future research agenda. International Review on Public and Nonprofit Marketing 13 (3): 289–317. https://doi.org/10.1007/s12208-015-0150-7 .

Article   Google Scholar  

Aitken, R., and A. Campelo. 2011. The four Rs of place branding. Journal of Marketing Management 27 (9–10): 913–933. https://doi.org/10.1080/0267257X.2011.560718 .

Anholt, S. 2007. Competitive identity: The new brand management for countries, regions and cities . Basingstoke: Palgrave-Macmillan.

Book   Google Scholar  

Anholt, S. 2010. Definitions of place branding—Working towards a resolution. Place Branding and Public Diplomacy 6 (1): 1–10. https://doi.org/10.1057/pb.2010.3 .

Ashworth, G.J., and M. Kavaratzis (eds.). 2010. Towards effective place brand management: Branding european cities and regions . Cheltenham: Edward Elgar.

Google Scholar  

Atlas.ti. 2018. Rigor in qualitative & quantitative scientfic research. http://atlasti.com/rigor-social-science-research/ . Accessed 11 Dec 2017.

Baker, B. 2007. Destination branding for small cities: The essentials for successful place branding . Portland: Creative Leap Books.

Baxter, P., and S. Jack. 2008. Qualitative case study methodology: Study design and implementation for novice researchers. The Qualitative Report 13 (4): 544–559.

BPS. 2018. Statement on qualitative research in journals | BPS. https://www.bps.org.uk/news-and-policy/statement-qualitative-research-journals . Accessed 9 May 2018.

Bryman, A., and E. Bell. 2007. Business research methods , 2nd ed. Oxford: Oxford University Press.

Charmaz, K. 2006. Constructing grounded theory . Thousand Oaks: Sage Publications.

Cresswell, T. 2004. Place: a short introduction . Malden: Blackwell Pub.

Cresswell, T. 2009. Place. In International encyclopedia of human geography , 1st ed, ed. R. Kitchin and N.J. Thrift, 169–177. Amsterdam: Elsevier.

Chapter   Google Scholar  

Dul, J., and T. Hak. 2007. Case study methodology in business research . Boston: Elsevier.

Dumez, H. 2015. What is a case, and what is a case study? Bulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique 127 (1): 43–57. https://doi.org/10.1177/0759106315582200 .

Dyer, W.G., and A.L. Wilkins. 1991. Better stories, not better constructs, to generate better theory: A rejoinder to Eisenhardt. Academy of Management Review 16 (3): 613–619.

Eisenhardt, K.M. 1989. Building theories from case study research. Academy of Management 14 (4): 532–550.

Eriksson, P., and A. Kovalainen. 2008. Research philosophy. Qualitative methods in business Research . https://doi.org/10.4135/9780857028044 .

Gibbert, M., W. Ruigrok, and B. Wicki. 2008. What passes as a rigorous case study? Strategic Management Journal 29 (13): 1465–1474. https://doi.org/10.1002/smj.722 .

Given, L., and K. Saumure. 2008. Rigor in qualitative research. In The SAGE encyclopedia of qualitative reseach methods , vol. I, ed. L. Given, 795–796. Thousand Oaks: SAGE.

Greene, D., and J.L. David. 1984. A research design for generalizing from multiple case studies. Evaluation and Program Planning 7 (1): 73–85. https://doi.org/10.1016/0149-7189(84)90027-2 .

Hankinson, G. 2007. The management of destination brands: Five guiding principles based on recent developments in corporate branding theory. Journal of Brand Management 14 (3): 240–254. https://doi.org/10.1057/palgrave.bm.2550065 .

Hankinson, G. 2009. Managing destination brands: Establishing a theoretical foundation. Journal of Marketing Management 25 (1/2): 97–115. https://doi.org/10.1362/026725709X410052 .

Hankinson, G. 2010. Place branding theory: A cross-domain literature review from a marketing perspective. In Towards effective place brand management: Branding european cities and regions , ed. G.J. Ashworth and M. Kavaratzis, 15–35. Cheltenham: Edward Elgar.

Hanna, S., and J. Rowley. 2011. Towards a strategic place brand-management model. Journal of Marketing Management 27 (5/6): 458–476. https://doi.org/10.1080/02672571003683797 .

Hillebrand, B., R.A.W. Kok, and W.G. Biemans. 2001. Theory-testing using case studies. Industrial Marketing Management 30 (8): 651–657. https://doi.org/10.1016/S0019-8501(00)00115-2 .

Host, M., and P. Runeson. 2007. Checklists for software engineering case study research. In First international symposium on empirical software engineering and measurement (ESEM 2007) , pp. 479–481. IEEE.

Johnston, W.J., M.P. Leach, and A.H. Liu. 1999. Theory testing using case studies in business-to-business research. Industrial Marketing Management 28 (3): 201–213. https://doi.org/10.1016/S0019-8501(98)00040-6 .

Kavaratzis, M. 2004. From city marketing to city branding: Towards a theoretical framework for developing city brands. Place Branding and Public Diplomacy 1 (1): 58–73. https://doi.org/10.1057/palgrave.pb.5990005 .

Kavaratzis, M., and G. Ashworth. 2008. Place marketing: How did we get here and where are we going? Journal of Place Management and Development 1 (2): 150–165. https://doi.org/10.1108/17538330810889989 .

Lillis, A.M., and J. Mundy. 2005. Cross-sectional field studies in management accounting research—Closing the gaps between surveys and case studies. Journal of Management Accounting Research 17 (1): 119–141. https://doi.org/10.2308/jmar.2005.17.1.119 .

Lucarelli, A., and P.O. Berg. 2011. City branding: A state-of-the-art review of the research domain. Journal of Place Management and Development 4 (1): 9–27. https://doi.org/10.1108/17538331111117133 .

Ma, W., D. Schraven, M. de Bruijne, M. de Jong, and H. Lu. 2019. Tracing the origins of place branding research: A bibliometric study of concepts in use (1980–2018). Sustainability 11 (11): 2999. https://doi.org/10.3390/su11112999 .

Miles, M.B., and A.M. Huberman. 1994. Qualitative data analysis: An expanded sourcebook , 2nd ed. Thousand Oaks: Sage Publications.

Mills, M.C. 2008. Comparative research. In The SAGE encyclopedia of qualitative research methods , vol. I, ed. L.M. Given, 68–71. Thousand Oaks: SAGE.

Mills, A.J., G. Durepos, and E. Wiebe (eds.). 2010. Encyclopedia of case study research . Los Angeles: SAGE Publications.

Nowell, L.S., J.M. Norris, D.E. White, and N.J. Moules. 2017. Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods 16 (1): 1609406917733847. https://doi.org/10.1177/1609406917733847 .

Ritchie, J.B.R., and R.J.B. Ritchie. 1998. The branding of tourism destination: Past achievements and future trends, in Destination marketing—Scope and limitations. In Reports of 48thn Congress , St-Gall: AIEST, 89–116.

Saunders, M.N.K., P. Lewis, and A. Thornhill. 2007. Research methods for business students , 4th ed. New York: Prentice Hall.

Shah, S.K., and K.G. Corley. 2006. Building better theory by bridging the quantitative? Qualitative divide. Journal of Management Studies 43 (8): 1821–1835. https://doi.org/10.1111/j.1467-6486.2006.00662.x .

Stake, R.E. 2006. Multiple case study analysis . New York: The Guilford Press.

Stoecker, R. 1991. Evaluating and rethinking the case study. The Sociological Review 39 (1): 88–112. https://doi.org/10.1111/j.1467-954X.1991.tb02970.x .

Tharenou, P., R. Donohue, and B. Cooper. 2007. Management research methods . New York: Cambridge University Press.

Vuignier, R. 2016. Place marketing and place branding: A systematic (and tentatively exhaustive) literature review. Working paper de l’IDHEAP , 5.

Vuignier, R. 2017. Place branding & place marketing 1976–2016: A multidisciplinary literature review. International Review on Public and Nonprofit Marketing 14 (4): 447–473. https://doi.org/10.1007/s12208-017-0181-3 .

Warnaby, G., and D. Medway. 2013. What about the ‘place’ in place marketing? Marketing Theory 13 (3): 345–363. https://doi.org/10.1177/1470593113492992 .

Whittemore, R., S.K. Chase, and C.L. Mandle. 2001. Validity in qualitative research. Qualitative Health Research 11 (4): 522–537. https://doi.org/10.1177/104973201129119299 .

Yin, R.K. 1981. The case study crisis: Some answers. Administrative Science Quarterly 26 (1): 58. https://doi.org/10.2307/2392599 .

Yin, R.K. 2009. Case study research. Design and methods , 4th ed. Thousand Oaks: SAGE Publications.

Download references

Acknowledgements

We would like to thank dr Renaud Vuignier for sharing his journal database and the reviewers of this journal for their insightful and constructive comments towards improving our manuscript.

Funding was provided by Cracow University of Economics.

Author information

Authors and affiliations.

Department of Public Management, Cracow University of Economics, ul. Rakowicka 27, 31-510, Kraków, Poland

Marek Ćwiklicki & Kamila Pilch

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Marek Ćwiklicki .

Ethics declarations

Conflict of interest.

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Ćwiklicki, M., Pilch, K. Multiple case study design: the example of place marketing research. Place Brand Public Dipl 17 , 50–62 (2021). https://doi.org/10.1057/s41254-020-00159-2

Download citation

Revised : 07 September 2019

Published : 05 February 2020

Issue Date : March 2021

DOI : https://doi.org/10.1057/s41254-020-00159-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Methodological rigour
• Place marketing
• Multiple case study
• Find a journal
• Publish with us
• Track your research

Home > JCPS > Vol. 15 (2022) > No. 2

Journal of Counselor Preparation and Supervision

A systematic approach to multiple case study design in professional counseling and counselor education.

Charmayne R. Adams , University of Nebraska at Omaha Follow Casey A. Barrio Minton , University of Tennessee Follow Jennifer Hightower , Idaho State University Follow Ashley J. Blount , University of Nebraska at Omaha Follow

Document Type

Case study, multiple case study, qualitative research, research design, counseling

Subject Area

Counseling, Counselor Education, Higher Education Counseling, Mental Health Counseling, Rehabilitation Counseling, School Counseling

Case study research is a qualitative methodology that allows researchers to explore complex phenomena in a structured way, that is rigorous and provides an enormous amount of depth. Three scholars are credited with major contributions to the case study literature: Merriam (1998), Stake (1995/2006), and Yin (1994). The purpose of this paper is to explore case study design for use in the counseling profession. The authors provide instruction on the case study scholars, data collection, analysis, and reporting for both single and multiple case study research designs. Finally, implications for student counselors, counselor educators, and counseling professionals are provided.

Recommended Citation

Adams, C. R., Barrio Minton, C. A., Hightower, J., & Blount, A. J. (2022). A Systematic Approach to Multiple Case Study Design in Professional Counseling and Counselor Education. Journal of Counselor Preparation and Supervision, 15 (2). Retrieved from https://digitalcommons.sacredheart.edu/jcps/vol15/iss2/24

Since June 01, 2022

To view the content in your browser, please download Adobe Reader or, alternately, you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.

• Journal Home
• About this Journal
• Aims & Scope
• Editorial Board
• Submit Article
• Most Popular Papers

Advanced Search

ISSN: 2164-3288

Home | About | FAQ | My Account | Accessibility Statement

Privacy Copyright

• Open access
• Published: 02 April 2024

Adopting, implementing and assimilating coproduced health and social care innovations involving structurally vulnerable populations: findings from a longitudinal, multiple case study design in Canada, Scotland and Sweden

• Gillian Mulvale   ORCID: orcid.org/0000-0003-0546-6910 1 ,
• Jenn Green 1 ,
• Glenn Robert 2 , 5 ,
• Michael Larkin 3 ,
• Nicoline Vackerberg 4 , 5 ,
• Sofia Kjellström 5 ,
• Puspita Hossain 6 ,
• Sandra Moll 7 ,
• Esther Lim 8 , 9 &
• Shioma-Lei Craythorne 3  

Health Research Policy and Systems volume  22 , Article number:  42 ( 2024 ) Cite this article

Metrics details

Innovations in coproduction are shaping public service reform in diverse contexts around the world. Although many innovations are local, others have expanded and evolved over time. We know very little, however, about the process of implementation and evolution of coproduction. The purpose of this study was to explore the adoption, implementation and assimilation of three approaches to the coproduction of public services with structurally vulnerable groups.

We conducted a 4 year longitudinal multiple case study (2019–2023) of three coproduced public service innovations involving vulnerable populations: ESTHER in Jönköping Region, Sweden involving people with multiple complex needs (Case 1); Making Recovery Real in Dundee, Scotland with people who have serious mental illness (Case 2); and Learning Centres in Manitoba, Canada (Case 3), also involving people with serious mental illness. Data sources included 14 interviews with strategic decision-makers and a document analysis to understand the history and contextual factors relating to each case. Three frameworks informed the case study protocol, semi-structured interview guides, data extraction, deductive coding and analysis: the Consolidated Framework for Implementation Research, the Diffusion of Innovation model and Lozeau’s Compatibility Gaps to understand assimilation.

The adoption of coproduction involving structurally vulnerable populations was a notable evolution of existing improvement efforts in Cases 1 and 3, while impetus by an external change agency, existing collaborative efforts among community organizations, and the opportunity to inform a new municipal mental health policy sparked adoption in Case 2. In all cases, coproduced innovation centred around a central philosophy that valued lived experience on an equal basis with professional knowledge in coproduction processes. This philosophical orientation offered flexibility and adaptability to local contexts, thereby facilitating implementation when compared with more defined programming. According to the informants, efforts to avoid co-optation risks were successful, resulting in the assimilation of new mindsets and coproduction processes, with examples of how this had led to transformative change.

Conclusions

In exploring innovations in coproduction with structurally vulnerable groups, our findings suggest several additional considerations when applying existing theoretical frameworks. These include the philosophical nature of the innovation, the need to study the evolution of the innovation itself as it emerges over time, greater attention to partnered processes as disruptors to existing power structures and an emphasis on driving transformational change in organizational cultures.

Peer Review reports

Growing recognition by governments internationally of the need to involve the perspectives of people using public services when designing, delivering and improving those services has been described as a Participatory Zeitgeist reflecting the “spirit of our time” [ 1 , 2 (p247)]. Researchers and designers have developed various approaches drawn from different disciplines and using different labels (for example, codesign, cocreation, coproduction) that align with principles in the citizen engagement literature [ 3 ]. These approaches recognize that service users have experiences and assets and can contribute to service design and delivery along with professional expertise, rather than simply being passive recipients of services designed and delivered by others [ 3 , 4 ]. While these approaches can be used with anyone, they have been increasingly applied to promote the inclusion of structurally vulnerable populations in the design and delivery of innovative health and social care services that seek to support them.

While coproduction has the potential to reform inequitable structures and social processes, excluding vulnerable groups or involving them in a tokenistic manner may unintentionally reinforce existing power imbalances [ 4 , 5 ]. For example, gaps have been noted between the rhetoric of service user involvement in international mental health policy and the readiness to adopt such policies in practice [ 6 ]. Challenges have also been noted in incorporating the voices of individuals with complex needs in improving care coordination across health and social services [ 7 ].

Despite increasing attention to coproduction in the literature and practice, knowledge gaps exist with respect to the implementation of coproduction involving vulnerable populations in different contexts [ 8 , 9 , 10 ]. An international symposium of coproduction researchers and people with lived experience held in Birmingham, England in 2017 identified the need for research to understand how exemplary coproduction innovations involving structurally vulnerable groups originated and their assimilation into routine practice [ 11 ]. To our knowledge, established implementation science models have yet to be applied to coproduction, where service users and service providers are cocreating innovations during the process of implementation [ 12 ].

In this paper, we present findings from a longitudinal case study exploring the factors and processes that influence the adoption, implementation and assimilation of three diverse coproduced public service innovations involving structurally vulnerable groups. We explored the perspectives of strategic leaders involved in advancing coproduction processes involving vulnerable groups. Our analysis proceeds through the lens of existing frameworks from the literature to discuss the outer context (economic, social, political, geographical), inner context (organizational and community considerations), individual factors, innovation features and process considerations [ 13 ].

Conceptual foundations: coproduction, structural vulnerability and implementation processes

Coproduction: Coproduction has been defined as “… involvement of public service users in the design, management, delivery and/or evaluation of public services” [ 4 ]. A core feature of coproduction approaches is that they are applied in a flexible manner, dynamically and innovatively responding to local needs and context [ 14 ].

Structural vulnerability: We adopt the term structurally vulnerable populations to recognize that vulnerability is not inherent in these populations but rather in the social, economic and political systems in which they are embedded [ 15 , 16 ]. Examples include individuals who may require multiple health and/or other public services, including people with complex and intersecting health needs (for example, heart failure and dementia) along with poverty, homelessness and/or being members of newcomer or racialized groups. Structural barriers (for example, lack of trust, language, cultural, scheduling, financial) and power relations may prevent them from engaging in coproduction.

Adoption, implementation and assimilation: We draw on and combine elements from three theoretical frameworks to guide this research. The first is the Diffusion Of Innovation (DOI) model [ 17 ], which identifies how political, social, economic, cultural, and organizational factors and processes affect fidelity and adoption during the diffusion of service innovation. The second is the Consolidated Framework for Implementation Research (CFIR) [ 18 , 19 ], which demonstrates the importance of contextual factors at multiple levels (external context, internal context, innovation features, processes and individual characteristics) in shaping the implementation of service improvements. The third is Lozeau et al.’s (2002) compatibility gaps [ 20 ], which characterize different forms of assimilation of innovations into routine practice [ 20 , 21 ]. Based on these frameworks, we define innovation as a novel set of behaviours, routines and ways of working that are directed at improving health outcomes, administrative efficiency, cost effectiveness or users’ experiences, and that are implemented by planned and coordinated action [ 20 ]. We define adoption as the incremental considerations and progressive individual and collective decision-making from pre-contemplation through exploration by which organizations ultimately decide to adopt the innovation (programme/model/process). Implementation describes the formal strategies to promote the integration of innovations into existing practices. Assimilation is the informal process by which, over time, innovations become part of routine ways of doing things. Assimilation can be characterized as (a) transformation when there is high fidelity to the model and the organization adjusts its functioning to fit the assumptions of the model; (b) customization when the model is adapted to the context and the organization adjusts its practices; (c) loose coupling whereby the innovation is adopted only superficially, while the functioning of the organization remains largely unaffected; or (d) co-optation whereby the innovation becomes captured and distorted to reinforce existing organizational roles and power structures [ 21 ].

Study aim and design

We adopt a longitudinal multiple case study approach to understand the dynamic nature by which three coproduced innovations intended to address the needs of vulnerable populations were adopted, implemented and assimilated [ 22 ]. Case study research is well suited to studying contemporary phenomena in their real-life contexts, and theory is often adopted to focus the analysis, allowing the theory to be augmented or revised based on emerging findings [ 22 ]. To meet the criteria of being a ‘case’, an innovation had to be underpinned by a coproduction model involving structurally vulnerable populations in the design, management, delivery and/or evaluation of a public service that has advanced through these phases. Concepts from the CFIR, DOI and assimilation frameworks described above informed the case study protocol, semi-structured interview questions, data extraction and coding.

Case selection

The three cases were selected through the networks of the investigators to illustrate how coproduction involving vulnerable populations can be advanced in different contexts: the region of Jönköping, Sweden striving for better patient outcomes and experiences by tailoring care to the needs of people with multiple complex needs (Case 1 – ESTHER); the city of Dundee, Scotland aiming to advance the recovery of people with mental illness through greater collaboration with those with lived experience and among service organizations (Case 2 – Making Recovery Real [MRR]); and a rural and an urban branch of a national community mental health organization in a Canadian province that adapted the English Recovery College model of coproduced educational programming to support the recovery of people with serious mental illness (Case 3 – Canadian Mental Health Association [CMHA] Manitoba and Winnipeg and CMHA Central branches’ Learning Centres in Manitoba, Canada) (see Tables  1 , 2 and 3 ).

The study team were familiar with each of these cases and were confident in having good access to them over time. Additionally, their different national contexts offered the opportunity to consider macro-level factors. While each of these countries’ health and social care systems are largely publicly funded, funding is the responsibility of different levels of government (municipal, provincial and/or national) and services are administered and delivered primarily by local governments and/or designated authorities (see Table  4 ).

Data sources and collection

Data sources include relevant academic and grey literature identified through electronic searches and/or recommended or shared by local gatekeepers and key informants to inform the background case context for the individual case analyses, and the interview guides (see Table  5 , and Table S1 in Additional file 1 for more details). Research team members (GM, JG, GR, NV, PH, SC, SS) conducted 45–60 minute long semi-structured interviews in person or online between November 2019 and August 2021. To help understand the history and context of each case, key informants were strategic decision-makers and programme managers affiliated at the time with the organizations leading, participating in or supporting the local initiatives, and who were familiar with the history of how the coproduced innovations emerged, their developmental timeline and coproduction’s role in the overall system. Footnote 1

The interview guide questions probed about this history with a focus on the contextual factors that influenced adoption and implementation and the extent to which coproduction has been assimilated into routine practice. Data were gathered through investigator field notes, the audio-recording and transcription of interviews, timelines, hand-written notes and/or audio-recordings of team meetings to capture member checking with local collaborators, and case team memos of decision points.

To maintain participant anonymity, participant codes are used in the text, identified by a location code (for Case 1, JKG = Jönköping, Sweden; for Case 2, DND = Dundee, Scotland; for Case 3, OTH = Other [for example, national, international informants], PLP = Portage la Prairie, Manitoba, Canada; WPG = Winnipeg, Manitoba, Canada), and a participant number (that is, 01, 02, 03 and so on). For example, an informant from Dundee could be DND-03. Note that the perspectives of service providers and people with lived experience of structural vulnerability were not the focus here but are considered in subsequent waves of our data collection to understand their experiences of coproduction in practice.

Data analysis

A common coding framework was developed iteratively to capture factors and processes influencing adoption, implementation and assimilation by combining elements of the theoretical frameworks to remove overlap and promote consistency of understanding when coding and interpreting the data. Table S2 presents this in more detail (see Additional file 2 ).

The initial data extraction was performed by the research team affiliated with each case, and the project research coordinator worked with the local research coordinator for each case to ensure consistency across cases. Documentary evidence analysis primarily informed our understanding of the historical context and overview of each case. All data were coded and analysed using a deductive approach; a common coding scheme and thematic analysis were employed, respectively, based on the theoretical propositions and concepts in the CFIR and DOI models, and allowing for emergent themes, particularly in relation to the coproduction context [ 22 ]. A visual timeline was created to understand the initiation and growth of coproduction in each case. Interview data was triangulated with documentary evidence and field notes. Analysis proceeded on a case-by-case basis, followed by a cross-case analysis.

Qualitative validity and reliability

The research team comprised four members who were familiar with one of the three cases prior to the study (the ESTHER case), as well as eight members who were not familiar with any of the cases. One member of the team had been closely involved with the development of the ESTHER case over a long period of time. The use of a common and detailed case study protocol and data management system, central and local research coordination by case, monthly investigator meetings and tri-annual full team meetings including collaborating organization representatives were strategies used to enhance qualitative validity. The common coding framework and frequent team discussions helped to ensure consistency and enhanced reliability. Data were triangulated across sources, the analysis was triangulated across investigators and theories, and member checked at various stages with the full team of investigators and collaborators [ 23 ].

Ethical considerations

Research ethics clearance was obtained from the relevant academic research ethics boards (McMaster University Research Ethics Board [MREB Project ID 2066], Aston University Ethics Committee [Rec Ref #1611]; King’s College London Research Ethics Office [Reference Number MOD-19/20-17350]; SingHealth Centralized Institutional Review Board [CIRB Ref# 2020/2341]; and Swedish Ethical Review Authority [Etikprövningsmyndigheten, Dnr 2019-06373]), and in light of this, ethics review was waived by the boards of the collaborating organizations (Canadian Mental Health Association, Manitoba & Winnipeg branch, the East of Scotland Research Ethics Service). Participants received letters of information outlining the study objectives, protocol and risks prior to consenting in writing. Data were collected and stored locally and shared across sites as anonymized, encrypted and password-protected files.

We outline the historical context and analysis of contextual factors influencing adoption and implementation, discuss assimilation by case and then present a cross-case analysis. Tables 1 , 2 and 3 above capture the key features of each case, Figs.  1 , 2 and 3 summarize the adoption, implementation and assimilation timelines, and Tables  6 , 7 and 8 summarize the cross-case analysis.

Historical context: ESTHER is a complex system of public health and social care services run by 13 municipal councils in Region Jönköping County, Sweden that has brought intersectoral health and social care providers together since the 1990s to increase coordination and to redefine service experiences around the needs of the person receiving the services. In a context of restricted public sector funding, ESTHER began in 1997, initially for 2 years, with the aim of finding ways to meet population health needs using approaches other than increased hospital bed capacity. Hospital leaders in Region Jönköping County aimed to transform ways of working and to prevent hospital admissions through what informants called “radical customization”, which considered the needs of individual patients using a bottom-up change process referred to as health process re-engineering. This approach 'shadowed' a patient with complex needs through their health service experience journey and included interviews and surveys with patients, staff and government officials and observations of care encounters and processes to gain new insights into what was needed to improve the system from the patient perspectives. Storytelling of the experience of 'Esther', a persona of an elderly person with complex health needs, actualized this process, pointing out what needed to be done differently by demonstrating the importance of focusing on the experience of the person receiving care. The lessons learned from ESTHER fuelled health and social service-wide change, including coproduction with patients beginning in 2006 through patient roles on advisory committees and councils, and has expanded to include initiatives such as ESTHER cafes, ESTHER coach training and ESTHER family meetings, among others.

Adoption: In the ESTHER case 'adoption' of coproduction was an emergent phenomenon that took place over a 10 year period as ongoing improvement efforts, aimed from the outset at better capturing the lived experience of people with complex needs, evolved in terms of how their perspectives were incorporated in design and decision-making. This initially began with interviews and shadowing patients and bringing staff on board with this approach, until by 2006, Esthers became more directly involved in coproducing system improvements. In the internal context , healthcare process re-engineering efforts since the 1990s centred on the question of “What is best for Esther?” and demonstrated the importance of person-centred care and emphasizing the experiences of the person in need of complex care, laying the foundation for a coproduction approach to emerge. In the external context , system-wide efforts by health and social leaders to create a system map led to ESTHER becoming more than a health quality improvement project but rather a health and social systems-wide movement. From a process perspective, the initial project’s evaluation results indicated a 20% reduction in hospital beds, an achievement that earned recognition in the external context through two national awards. As project funding ended, the benefits of the ESTHER philosophy were recognized, and ESTHER transitioned from a project to a 'network' without funding. Over the next few years, the ESTHER Network further developed as 'cousins' emerged across Sweden, and the approach was adopted in other countries, including Italy, England, Scotland and France.

By 2006, ESTHER in Sweden transitioned toward adopting coproduction approaches that actively invited participation of people with lived experience expertise (Esthers) in coproducing ongoing innovations; however, this process was emergent and not uniform. The flexibility of a guiding philosophy was a key feature that enabled this emergence of innovation in the coproduction approach. By this time, some individual system leaders had come to recognize that keeping the focus on value and what is best for the person being treated in their daily lives would lead to better results than a preoccupation with resources and cost cutting. ESTHER had transformed relationships internally in hospitals to team-based (doctor‒nurse) coleadership and externally across the region via interorganizational collaboration between hospitals, primary care, community care and social care to improve Esthers’ care journeys. These collaborative ways of working were preparation for collaboration with Esthers, helping to create receptivity among senior leaders to coproduction. Nonetheless, at this stage of adoption there was still some internal resistance, particularly at middle management and staff levels, as Esthers began attending and sharing stories about their experiences at leadership meetings.

“I think one of the most important decisions was to take patient in the room. In addition, there was a lot of resistance”. [JKG-01]

Implementation: Once the decision to work directly with Esthers was taken, the implementation of coproduction has continued to unfold, albeit unevenly and opportunistically. Around this time, factors in the outer context shaped ESTHER’s continued development, as Esthers became increasingly present in local patient committees and began to participate in and influence the ESTHER steering committee. While ongoing primary care reform was a distraction for many health service managers, an external network of Esthers developed from different programmes across municipalities, and annual ESTHER 'family' meetings were held, where Esthers could convene to share experiences and ideas, strengthening the grassroots support. ESTHER was again gaining international recognition, becoming the subject of a BBC documentary film and being declared “one of the coolest innovations in the world” by CNN.

In the internal context , further developments included the creation of internal structures that were funded to support greater involvement of patients with multiple vulnerabilities in coproduction activities: The ESTHER Competence Center, training healthcare teams to follow the ESTHER philosophy, and ESTHER Coach quality improvement training programmes for approximately 30 health and social service providers to become new ESTHER Coaches each year, and with growing numbers of Esthers as faculty. Key features of the approach were supportive of grassroots growth. Coaches developed innovations on an ongoing basis with input from Esthers, and health and social service providers remarked that the ESTHER philosophy takes them back to the reasons they entered their professions. At the same time, the bottom-up nature driving innovation continued to be threatening to some individuals in senior leadership positions who were more distanced from observing the benefits.

“ESTHER is very much bottom-up. So, you are very close to ESTHER … you see what’s going on and what you can do better. The steering is from the bottom, and then the managers got a bit threatened. I think there was suddenly too much; the movement was suddenly too big. So, people were reacting to that. …That still is a challenge”. [JKG-01]

Creative approaches have been used to foster growth despite this resistance. Small changes such as renaming committees have enabled participation by Esthers.

“We had our ESTHER Strategy Days. It was once a year that we had a really big gathering about what we are going to focus on. And we invited managers, we invited the coaches, we invited Esthers. So, one-third of the group [of 30] were Esthers and the other were working in health and social care. And, for me, that was a very big success, but it also became a threat. So, they took it away because they said you can’t have strategy day because you are not a manager. So, we changed the name. Now we have the ESTHER Inspiration Day”. [JKG-01]

The implementation process has been incremental and iterative to balance the grassroots pressure for innovation with the internal resistance to patients as equal partners, while ensuring real change results. As an example, in 2007, ESTHER cafes were introduced to connect Esthers and to identify the improvement possibilities most important to Esther. These cafes continue to be held four times per year and have attracted a wide audience, including clinicians and politicians. Esthers share their stories to help leaders and practitioners understand individual experiences, but the process also builds credibility: it requires a check-in with leaders and service providers about what they heard and whether that is consistent with what the storyteller feels is most important, and agreements are reached before the meeting ends about specific action(s) that will be taken to address what is important to Esthers.

“When we listen to a story, we ask the group, ‘What did you hear?’ And we are trying to confirm whether we are hearing different things than [what] Esthers really mean. So, the staff sometimes think, ‘This is very important’. But when we give that back to Esther, she says, ‘Well, that’s not so important for me. For me, this is important’. So, the ESTHER cafe is an activity to identify improvement possibilities. That’s one of the activities”. [JKG-02]

Assimilation: By 2016, ESTHER had evolved from being a network to becoming assimilated as a mindset – the central concept driving innovation in the system in the Jönköping Region. By this time, the decision was made to withdraw funding specific to ESTHER other than to support coach education and to have no single person responsible as leader, as it is intended to be fully assimilated as part of the normal way of working. At the same time, without dedicated funding and leadership, questions remain about sustainability.

“As I said, it is a mindset. Now it is implemented in these programs – the question: ‘What’s best for Esther?’– you will find you can’t find one person who is responsible for ESTHER in Sweden, but there is a programme group and the programme group is trying to find out ways how to spread it in the whole region, because we have some difficulties there. It’s a mindset and it should be part of the daily work. And we are getting there. I think it’s very much dependent who is leading all these kind of leadership programmes, and do they really take the ESTHER philosophy to heart?” [JKG-02]

At this point, all steering groups were removed, being seen as no longer necessary. This removal of infrastructure (formal structures, funding) initially concerned committed leaders, with a risk of co-optation of the ESTHER concept without true adherence in practice. However, there was a widespread sense among interviewees that the ESTHER philosophy has been assimilated as a core value that continues to influence all activities, permeating the culture to become the routine practice in Jönköping.

“It’s a very normal mindset in one of our hospitals to ask the question, ‘What’s best for Esther?’ That’s just a normal way of working and people are just using that word and that question”. [JKG-FL-01]

See Fig. 1 for a summary of the Case 1 adoption, implementation and assimilation timeline.

Case 1 ESTHER coproduction adoption, implementation and assimilation timeline

Historical context: Making Recovery Real gives people with lived experience of mental health difficulties the opportunity to be at the centre of decision-making, service design and practice development in the community of Dundee, Scotland by changing the terms of the dialogue about recovery, mental health and well-being. It began in 2015 as a collaboration of 10 local public, voluntary and community organizations who responded to a call from the Scottish Recovery Network (SRN) to work together to take a new approach to improve the experience and outcomes for people living with mental illness. Initially, the partner organizations endeavoured to develop and deliver more recovery-focused policies and practice by centring lived experience in answering the question: “How can we make recovery real in Dundee?” They brought together interested people, including those with lived experience, at collaborative cafes; a series of events where priorities and accompanying actions were identified, and where participants were equal contributors to the process and its outcomes. To foster the integration of lived experience into system design and practice, the priorities identified were to (i) collect and share recovery stories so that lived experience is at the core of service design, delivery and practice; (ii) develop peer support roles and training; and (iii) celebrate recovery [ 24 ].

Adoption: In the external context , the mental health system remained dominated by the medical model, a lack of system innovation and acute services prioritized over community services. Yet, recent Scottish health and social care system integration has supported partnership working. Simultaneously, SRN, a national voluntary organization established in 2004 to promote recovery principles within the mental health system, was shifting from working with the National Health Service towards building coalitions of change within communities and a whole-systems approach to promoting recovery. SRN solicited proposals from local groups and organizations, offering their support for community-based collaborations that would involve people with lived experience in developing local initiatives to support mental health.

Factors in Dundee’s internal context also converged to support a proposal put forward to SRN for an innovative approach. First, the Dundee Third Sector Interface (TSI), which supports the representation of third sector organizations in local authority planning, had been working to better involve people with lived experience in mental health system planning, and meetings with their network members were becoming more recovery focused. A recent inquiry into mental health services and a fairness commission on poverty (a longstanding local issue) also motivated the local council and Health and Social Care Partnership (HSCP) to take innovative action focused on prevention versus mitigation.

“And I think the Health and Social Care Partnership realized that they needed to do more than mitigation … they have been really, really clear on the need for new ways of doing things for about the last 10, 15 years”. [DND-02]

Furthermore, Dundee City was preparing to develop a new mental health strategic plan and, in the hope of influencing the strategic priorities and the future approach to engagement locally, the TSI brought partners from across community services, the local authority and representative groups who had been attempting to make change in the system to submit a proposal for SRN’s support. Individual leaders from within the partner organizations, motivated by their own lived or professional experience, were drawn by the innovation’s features : to support any concerned citizen to contribute their inherent resources through meaningful involvement and an asset-based approach:

“… So lived experience is essential, bringing people together, involving everybody who wants to be involved in each aspect of the process; so, firstly in agreeing what it is they want to achieve, then in making sure that it is carried out, also in having an actual role in actively carrying it out, so not just identifying things other people should do but having a vested interest and an active contribution to the activities that are going to be – whatever it is that’s going to be done differently, basically”. [DND-04]

SRN acted as a change agency, helping to alleviate tensions among the coalition and supporting their process of exploring the opportunity and submitting a successful proposal.

Implementation: First, SRN helped to bring the individuals involved together to establish a shared vision for the process among the local integration bodies (TSI and HSCP) and a TSI-supported service user network, reducing competition among the service provider partners. Within the inner context of the partnership, there was a commitment to coproduction processes and peer support as a critical opportunity to incorporate more lived experience into the mental health system. Despite these efforts, some of the original partners could not align themselves with the experience-led approach and discontinued their involvement knowing they could return at any time. Undaunted, the remaining partners proceeded by working with the “willing”, beginning with increasing local knowledge of recovery approaches and exploring what recovery meant to local citizens.

“… at the very start, it was a case of, ‘Right. We don’t really know where we want this to go. And actually, are we the ones to be dictating where this should go? No, we’re not. What’s most important is that we’re listening to people with lived experience, people on the ground, and they should be the ones that are telling us what needs to be changing’. So from the beginning, the sort of first step was looking at how we can engage with local people. And we were really keen to make sure that it was meaningful … And we thought this involvement can’t be tokenistic. People need to be on board, and it needs to be collaborative from the start”. [DND-05]

To build connection and trust between participants while shifting to a peer-led approach, the implementation process involved facilitating a series of coproduced, discussion-based events where people with lived experience were invited to be involved in all stages from planning and executing the events, to identifying and achieving priorities. The role of professionals shifted to “being on tap, not on top” [DND-02]. SRN provided developmental support to the Dundee partners to deliver the events, the features of which were welcoming and inclusive, avoiding formal presentations in favour of fun, health-promoting activities that allowed community members to feel heard, and demonstrated alignment with their own ideas and values.

“… what we did—and I would say I think that really set the tone – was rather than have lots of presentations, what we did was, at the event, we welcomed everybody, but we invited lots of the groups to run taster sessions of the things they did. So, that actually brought a lot of people with lived experience because they were coming along to demonstrate their finger painting. There was hula-hooping. There was wellness action planning. There was how to sleep well [sessions]. And in every corner of this venue, there was little groups of people who were painting pebbles, things like that. And then in the afternoon, we had a big conversation happen, world café style. And the sort of comments we got from people were, ‘I felt this was my event. This was for me. It wasn’t for them, the professionals’”. [DND-02]

From these discussions, it emerged that understanding local experiences of personal recovery was the most preferred and effective conveyor of local knowledge and motivator for change for the range of stakeholders. Storytelling became the primary vehicle for relationship building. Peoples’ stories were compiled into a film that premiered at a well-attended, prestigious 'red-carpet' event at a local cinema house, and subsequently became a tool to foster collaborative conversations at engagement events.

“And the film galvanised things and I think because we’d moved beyond that individual telling their story to having a 20 minute film of people reflecting on recovery, which is quite different from telling a story, say, of illness”. [DND-02]

The film drew strategic attention to MRR. This culminated into a consensus to embed recovery, backing for continued peer support and recovery work into the new Dundee Mental Health Strategy and accompanying action plan.

Assimilation: The MRR partner organizations have adopted a peer-led approach to their efforts to promote mental health recovery going forwards. Partners are also now far more involved in collectively determining the distribution of funding through the HSCP and in designing new mental health services.

Locally, the MRR approach has also been included in the Dundee Mental Health Strategy, granting the third sector more influence and collective power in local health and social care planning. The adoption of the MRR approach by the Dundee HSCP has strengthened the importance of mental health locally, dovetailing with the recommendations of the independent inquiry on poverty. At the national level, a Scottish government funding programme to increase the number of mental health workers in community-based services provided an opportunity for the HSCP to fund additional peer support roles, a key initiative within MRR.

Overall, the MRR partnership can be said to have had a transformative effect locally. It has led to better working relationships between providers and continues to drive progress. Furthermore, lived experience is being built into the system infrastructure through actions prioritized in experience-centred collaborative conversations: expansion of the local peer recovery network, development of peer support roles, implementation of peer-led services, peer support training provision and building recovery awareness. A key feature of ongoing progress has been that lived experience partners have been able to move in and out of active participation roles throughout the process, as their recovery journeys and contexts have allowed.

“There was that sense of collaboration that continued ... We kind of all came together to discuss how we felt our organizations could contribute to that bigger picture and the strategic objectives moving forward, and not just the strategic objectives in relation to Making Recovery Real but the wider kind of city and what they were looking for in relation to the local mental health strategy and the city plan”. [DND-05]

Participants describe the process as a difficult yet joyful and rewarding journey. For some organizations, the introduction of the MRR approach has motivated significant recovery-oriented change in their values and structure, further cementing system-level impact.

“Making Recovery Real has really been – I suppose we’ve adopted the principles and approaches … We try to adopt those as far as possible in all of our work. And we don’t badge it all Making Recovery Real, but we use the learning from it, I would say, in everything we do now, everything in the programme”. [DND-04]

See Fig. 2 for a summary of the Case 2 adoption, implementation and assimilation timeline.

Case 2 Making Recovery Real coproduction adoption, implementation and assimilation timeline

Historical context: CMHA Learning Centres began development in Manitoba in 2015 as a coproduced adaptation and renaming of Recovery Colleges, which originated in England in 2009 with a focus on people with lived/living experience of serious mental illness. The aim of Recovery Colleges is to bring the lived experience of people with mental illness and other community members together with professional expertise to locally plan, develop and deliver educational courses about mental health and recovery, with the aim of empowering people to support their mental health and well-being. The concept of recovery education originated in the USA [ 25 , 26 ], and before adopting the Recovery College model, CMHA Winnipeg had offered psychosocial rehabilitation (PSR)-based recovery education since the early 1990s. In 2015, the CMHA Winnipeg branch leader conducted an internal evaluation of this programming, which suggested that improvement was needed to meet the psychosocial health and well-being needs of the community. Around the same time, the new leader of the CMHA Central branch in Portage la Prairie, Manitoba sought a fresh approach to its clubhouse programme, a mutual support drop-in centre, in response to member feedback. Leaders and service users of both branches embraced the Recovery College and coproduction approach to better meet client needs. CMHA Learning Centres build on the Recovery College principles, with the programming and the target audience expanded to promote living well among the broader population, as well as recovery education for people with lived experience of mental illness. The CMHA Central branch’s Thrive Learning Centre and the CMHA Winnipeg and Manitoba branch’s Well-being Learning Centre opened in September 2017 and January 2018, respectively.

Adoption: In the external context , the national policy context was supportive of a recovery and well-being approach; it was the focus of consultations over the 2008–2012 period prior to the release of Canada’s mental health strategy [ 27 ]. This enabled Manitoba bureaucrats to pressure provincial government leaders to cosponsor a 'Recovery Days in Mental Health' conference held in Winnipeg in June 2015. An English Recovery College champion was a keynote speaker and sparked interest in the model among CMHA branches in Manitoba. The Winnipeg Regional Health Authority (RHA), the major funder of the Winnipeg CMHA branch, also supported recovery and mental health promotion approaches. Informants reported that Manitoba’s culture of innovation and solidarity, with its many small rural communities, also aligned with the coproduction philosophy of inclusive innovation.

In the internal context , the Recovery College model resonated with existing branch cultures of deep commitment to recovery-oriented work and strong peer support foundations. CMHA’s federated structure allowed each branch autonomy to develop its own programming, with support from a national office. Attractive innovation features were the existing evidence base, emphasis on lived experience through coproduction in course development and facilitation, opportunity for student skill building, and flexibility to accommodate local needs and strengths. The instructional climate was also appealing, as it could offer people with lived experience a sense of community and could promote their self-efficacy and confidence while reducing the power imbalance and fostering relationships between staff and students. The Recovery College model could also offer a more immediate response in terms of educational support to people needing care and facing long wait times for traditional services.

“I would say there’s probably many other things besides instruction. I think there’s relationship-building that happens so there are connections between students and between the facilitators and the learners. It’s the development of a space that allows for people to develop skills that are unrelated to the content. So, people also learn skills like sharing in a group context, so confidence-building, self-efficacy. When you can cultivate a skill in one area, you build confidence, and you start to believe that you have the ability to learn and to develop new skills. So that sense of self-efficacy is very integral to the recovery and well-being journey”. [WPG-02]

The importance of individual characteristics was demonstrated as passionate leaders in the Winnipeg and Central branches who were committed to advancing upstream mental health promotion and PSR were impressed by the model and together, they researched it further to inform adoption decisions. The coproduction process aligned with CMHA’s “nothing about us without us” approach and could foster a sense of ownership. In both branches, the name Recovery College was changed to Learning Centre during the adoption process, which better resonated with community and agency participants.

Implementation: In the external context , in early 2017, CMHA Winnipeg and Central branches met with CMHA National to implement Learning Centres. Although no new funding was made available by the RHAs, philosophical support enabled the repurposing of existing funding for recovery education and peer support. In 2018, CMHA National and CMHA Winnipeg leadership visited England to meet recovery-focused mental health services experts and to see the model in action. This visit was crucial in fostering strong relationships between the model initiators and CMHA leaders who discovered common visions to widen the target audience to anyone in the community interested in mental health issues, thereby making mental health a universal concern and promoting a living well approach. Collaboration with an Ontario-based psychiatric hospital, with similar values and interest in Recovery Colleges, supported programme evaluation to produce evidence of effectiveness.

Internally , the Winnipeg and Central branches collaborated on initial model and course development, and took a staged approach to opening their Learning Centres. In the Central branch, where resources were tighter and there was a large geographic area to serve, creative approaches to leverage local support and assets were used. Health professional placement students supported the small branch to prepare for launch and in doing so, encouraged staff buy-in. Another peer service organization provided funding support and this, along with community grants, covered staffing, technology, social marketing and other costs that are traditionally not eligible for provincial funding.

“[A] critical moment would be the establishment of a partnership. I think that was a critical moment. I walked away and I know my staff did, too, with an immense sense of relief after I could tell them that [a peer Manitoban mental health community organization] was on board to help make this a reality”. [PLP-22]

The Winnipeg branch also leveraged internal resources, including an existing peer support group whose members assisted in developing the first five courses.

“And so we actually relied on some communities that existed within our CMHA. So we had a group of individuals who are peer supporters to one another. They had taken our workshops in the past. And then they created, on their own, their own support group, and designed that support group based on their needs and on an educational focus. So we actually asked them if they would be our initial coproduction group”. [WPG-04]

The passion of individual CMHA staff and leaders, many with their own lived experience, made them champions who demonstrated their commitment to valuing expertise derived from lived experience. These individuals also helped build the external linkages with organizations and key people both nationally and internationally. Innovation features allowed for initial small-scale implementation, leveraging local assets and community strengths before expanding further. The flexibility to offer “something for everyone” and promote “living well in your community” garnered broad interest and unanimous buy-in from community members. The flexibility of the model also allowed the Winnipeg branch to retain PSR influences from their colleagues at Boston College.

The collaborative coproduction process fostered a sense of ownership, friendship building, balance across perspectives and acceptance within the classroom. This affirming process allowed room for creative input and for trial and error, with the process itself evolving to become more effective over time. It also facilitated the expansion of course offerings, as students were encouraged to lead future course development. Accompanying changes to the physical space and staff roles helped in welcoming the whole community, meeting the needs of vulnerable groups in society and addressing access barriers.

Assimilation: The Central branch has been unable to coproduce new Learning Centre material during the COVID-19 pandemic, yet it continues to offer its existing content. The Winnipeg Learning Centre was able to shift to virtual and then hybrid online and in-person coproduction activities, while ensuring fidelity to the core Recovery College principles.

“And some of the other things that are in the fidelity assessment are: Are you recovery-focused? Are you community-focused? Are you collaborating with the people who are consuming your services? So, it’s a really easy fidelity to conform to but also have room to be kind of creative because they’re not dictating what courses you should have. The fidelity is that you provide courses”. [WPG-04]

In Winnipeg, the Learning Centre continues to expand and evolve, and is reported to have had a gradual but transformative impact on organizational context and values within the branch, by providing a universally accessible platform that demonstrates the value of engaging people with lived experience at every step. The coproduction approach to course development has ensured that content remains current and relevant through creativity, diversity and responsiveness to people’s needs. Leaders’ commitment to the model and ongoing evaluation to ensure it is meeting local needs have supported wider assimilation of coproduction approaches in other branch programming as well. New leadership in the Central branch has expressed the desire to revive the Learning Centre’s coproduction activities.

See Fig. 3 for a summary of the Case 3 adoption, implementation and assimilation timeline.

Case 3 CMHA Learning Centres coproduction adoption, implementation and assimilation timeline

Cross-case comparison

Adoption: Shifting ideas in the public policy realm and supportive external change agents created a conducive external context . In Cases 1 and 2, shifting ideas pertained to interprofessional and intersectoral collaboration and in Case 3, national and provincial discussions about a recovery and well-being orientation were important precursors to coproduction with people with lived experience. Internally , tension for change was evident in all cases; however, the process by which this unfolded differed, as a natural progression of ongoing improvement efforts in Cases 1 and 3 and as a deliberate response to an opportunity created by an external change agent for local system-wide transformative change in Case 2. In all cases, passionate individuals , many with their own lived experience, and a philosophical approach that resonated deeply and widely was a core feature leading to adoption (see Table  6 ).

Implementation: In all three cases, building local partnerships and/or networks in the external context was integral to implementation. These partnerships and networks helped to overcome internal resistance within existing power structures (Case 1), created a community coalition that could move forwards in the face of resistance within traditional mental health services (Case 2), and offered material support and expertise to support implementation (Case 3). In Cases 1 and 2, there was no 'programme' per se, rather a philosophy steered by guiding questions, and in Case 3, the Recovery College model itself was designed to realize its embedded philosophy through coproduced educational programming. These features drove a micro-level movement for change (all cases) that was locally adapted, for example, to become “something for everyone” (in Case 3). Philosophical alignment also helped in building trust across collaborating organizations to support implementation and as a shared foundation for overcoming differences during implementation. Implementation proceeded incrementally at the grassroots level in all cases and by working with the willing (see Table  7 ).

Assimilation: There have been different forms of assimilation across all three cases, with transformative impacts not only on the organizations involved but with impacts extending to the broader organizational and political context. A widely embraced mindset in the region, new structures and a growing international network (Case 1); impact on the local mental health strategy and continuing transformative effects on partnerships among community agencies (Case 2); and assimilation to other programmes and branches (Case 3) are some of the ongoing transformative impacts.

In Case 3, assimilation was characterized by customization, as both branches have changed the name and broadened the reach of Recovery Colleges, while maintaining fidelity to core principles. At the same time, challenges to sustaining such transformative change going forward were a concern without targeted leadership and funding (see Table  8 ).

The analysis of these cases of adoption, implementation and assimilation of innovation demonstrates a range of factors from existing frameworks that shaped the stories of these coproduced innovations. The analysis also suggests additional considerations beyond established frameworks when aiming to engage structurally vulnerable people in coproduction activities that can help to overcome structural barriers and address power differentials in legacy systems.

Existing frameworks and models were very helpful in pointing to the interplay between the many factors operating at different levels in each context. These comprehensive frameworks provided a wide lens that was useful for thoroughly investigating different contextual elements. However, at times, this comprehensiveness made it difficult to tease out the essential causal story from our data to understand how each set of coproduced innovations emerged [ 28 ]. In our analysis, existing frameworks were most helpful when comparing across cases to identify overarching patterns, such as the influence of shifting policy ideas and external change agents in the external context during adoption and the role of community partners and network building in the implementation phase.

At the same time, particularly compelling considerations involving structurally vulnerable groups identified here were less evident in existing frameworks. Notably, there were two important differences in the nature of the ‘programme’ in this context. First, existing frameworks suggest a predefined 'programme' to adopt; however, there was no predefined programme per se in two of our cases. Instead, change was more ideological/philosophical in nature, captured simply by a set of guiding questions (two cases) or embedded as a central feature of an existing program with lots of room for customization (one case). The central philosophy in these cases corresponded to efforts to raise the profile of traditionally marginalized voices by shifting normative paradigms about what types of knowledge (for example, lived experience) and whose voices (for example, structurally vulnerable service users) should be heard in traditional systems. Second, the process (coproduction) could not be disentangled from this essential philosophy and, in some cases, it was met with considerable resistance. Including vulnerable people as genuine partners in coproducing innovations was perceived as a 'threat' to some managers (Case 1) or to the prevailing orthodoxy of 'Quality Improvement' (Case 2).

These 'programme' features suggest a second consideration in terms of implementation processes . The clear intention to shift the existing power balance in systems and within organizations needed a set of resources that went beyond the capacity of any one organization. While high-level leaders with their own lived experience were instrumental in providing vision and support, the implementation process relied heavily on relationship building across partner organizations and networking at the grassroots levels rather than on top-down directives. Meaningful service user involvement was considered critical in making transformative service and system culture change, often disrupting traditional structures, networks and communication. Shared values, the development of a group-based belief system, core activities and a different relational environment and leadership [ 29 , 30 ] are central to social movement theories. Furthermore, the definitive objective of stepping outside organizations within the formal healthcare system to instead derive a new way of working across many community organizations led by people with lived experiences is not clearly captured in existing frameworks, which typically speak to innovation within existing structures of power in organizations and systems.

Finally, the cases analysed here suggest important differences in temporal dynamics at play that were not elaborated in existing models. Consistent with concepts of change in complex adaptive systems and theories of policy path dependence and agenda setting, adoption could occur through a slow internal tension for change that built over time and culminated in coproduction as a natural evolution of ongoing improvement efforts or through seemingly sudden 'transformative' reform where a confluence of interested groups came together in the face of an opportunity to do something differently. Ideas about change in complex adaptive systems such as emergence, self-organization, adaptation, change over time, distributed control and tipping points [ 31 ], and from policy literature such as path dependence [ 32 ], multiple streams theory [ 33 ] and distributed control could be informative in this respect [ 34 ]. Our participants suggested that because each case relates to a set of concepts and principles that were collectively generated over time, there was a need to better understand this process as it unfolded.

While existing models were helpful in considering a wide range of factors to consider and recent updates suggest a movement away from concepts such as 'programme' to 'innovation' [ 19 ], the temporal, relational and power dimensions discussed here were validated by our collaborators as equally important considerations. Exploring these dimensions will be the focus of future work.

Limitations and future work

This work is subject to several limitations. First, it is based on a case study of three examples of coproduction of health and social care innovations in different national contexts in the northern hemisphere. The findings may not be transferable elsewhere. Furthermore, when considering our findings in relation to the CFIR, DOI and assimilation frameworks, it is important to note that these frameworks were not specifically developed for an innovation process involving service users at all stages of innovation adoption, implementation and assimilation. However, the limitations in adopting and applying these frameworks here have led to a careful examination of what is unique to coproduction processes involving vulnerable populations. A forthcoming contribution will try to capture these unique elements and position them within the innovation, power, and social movement literatures. Finally, the analysis here is primarily based on our 'wave 1' home site findings from this longitudinal case study, and new insights may be gained from a deeper evaluation of our wave 2 and wave 3 findings. The latter pertain to processes of ongoing coproduction in practice and diffusion to other contexts, respectively, and will be analysed in forthcoming work.

While our case study was extremely helpful in identifying core considerations for factors influencing the adoption, implementation and assimilation of three cases of coproduced health and social care innovations, several nuanced considerations when applying existing theoretical frameworks in the coproduction context emerged: the nature of the 'intervention' being a philosophy rather than a concrete set of steps, the intertwining of intervention and process and the need to study evolution of the intervention itself as it emerges over time, greater attention to partnered processes as disruptors to existing power structures and an emphasis on driving transformational change in organizational cultures. Future work will explore these considerations further.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due to the study’s small sample size and the key informants’ roles as leaders within small organizations, making it difficult to deidentify their data. However, the datasets are available from the corresponding author upon reasonable request.

In some cases, these individuals also had lived experience of vulnerability that also motivated their work, but this was not a specific requirement for study participation.

Abbreviations

Consolidated Framework for Implementation Research

(SingHealth) Centralized Institutional Review Board

Canadian Mental Health Association

Diffusion of innovation

Health and Social Care Partnership

McMaster University Research Ethics Board

Making Recovery Real

Portage la Prairie

Psychosocial rehabilitation

Regional health authority

Scottish Recovery Network

Third sector interface

United States

Robert G, Locock L, Williams O, Cornwell J, Donetto S, Goodrich J. Co-producing and co-designing. In: Dixon-Woods M, Martin G, editors. Elements of improving quality and safety in healthcare. Cambridge: Cambridge University Press; 2022.

Google Scholar  

Palmer V, Weavell W, Callander R, Piper D, Richard L, Maher L, et al. The Participatory Zeitgeist: an explanatory theoretical model of change in an era of coproduction and codesign in healthcare improvement. Med Humanit. 2018;45:247-57.

Article   PubMed   PubMed Central   Google Scholar  

McGeachie M, Power G. Co-production in Scotland—a policy overview. 2017.

Osborne S, Radnor Z, Stokosch K. Co-production and the co-creation of value in public services. Public Manag Rev. 2016;18(5):645–59.

Article   Google Scholar  

Iedema R, Merrick E, Piper D, Britton K, Gray J, Verma R, et al. Codesigning as a discursive practice in emergency health services: the architecture of deliberation. J Appl Behav Sci. 2010;46(1):73–91.

Sandhu S, Priebe S, Leavey G, Harrison I, Krotofil J, McPherson P, et al. Intentions and experiences of effective practice in mental health specific supported accommodation services: a qualitative interview study. BMC Health Serv Res. 2017;17(1):1–13.

Vackerberg N, Andersson A-C, Peterson A, Karltun A. What is best for Esther? A simple question that moves mindsets and improves care. BMC Health Serv Res. 2023;23(1):873.

Brown PR, Head BW. Navigating tensions in co-production: a missing link in leadership for public value. Public Adm. 2019;97(2):250–63.

First Nations Information Governance Centre. The First Nations principles of OCAP® [Internet]. Akwesasne (ON): First Nations Information Governance Centre (FNIGC); [cited 2023 Nov 12]. Available from: https://fnigc.ca/ocap-training/

Fotaki M. Co-production under the financial crisis and austerity: a means of democratizing public services or a race to the bottom? J Manag Inq. 2015;24(4):433–8.

Mulvale G, Robert G. Special issue- engaging vulnerable populations in the co-production of public services. Int J Public Adm. 2021;44(9):711–4.

Williamson V, Larkin M, Reardon T, Pearcey S, Button R, Green I, et al. School-based screening for childhood anxiety problems and intervention delivery: a codesign approach. BMJ Open. 2022;12(6):e058089.

Nelson EC, Batalden PB, Godfrey MM. Quality by design: a clinical microsystems approach. 1st ed. San Francisco: Jossey-Bass; 2007.

Batalden M, Batalden P, Margolis P, Seid M, Armstrong G, Opipari-Arrigan L, et al. Coproduction of healthcare service. BMJ Qual Saf. 2016;25(7):509–17.

Article   PubMed   Google Scholar  

Grabovschi C, Loignon C, Fortin M. Mapping the concept of vulnerability related to health care disparities: a scoping review. BMC Health Serv Res. 2013;13(94):1–11.

Katz A, Hardy B-J, Firestone M, Lofters A, Morton-Ninomiya ME. Vagueness, power and public health: use of ‘vulnerable’ in public health literature. Crit Public Health. 2020;30(5):601-11.

Greenhalgh T, Robert G, MacFarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(50).

Damschroder LJ, Reardon CM, Opra Widerquist MA, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17(1):1–16.

Lozeau D, Langley A, Denis J-L. The corruption of managerial techniques by organizations. Human Relat. 2002;55(5):537–64.

Robert G, Sarre S, Maben J, Griffiths P, Chable R. Exploring the sustainability of quality improvement interventions in healthcare organisations: a multiple methods study of the 10-year impact of the ‘Productive Ward: Releasing Time to Care’ programme in English acute hospitals. BMJ Qual Saf. 2020;29:31–40.

Yin RK. Case study research and applications: design and methods. 6th ed. Thousand Oaks, California: SAGE Publications, Inc.; 2018.

Maxwell JA. Designing a qualitative study. In: Bickman L, Rog DJ, editors. The SAGE handbook of applied social research methods. 2nd ed. Thousand Oaks, California: SAGE Publications, Inc.; 2009. p. 214–53.

Sharp C. Making Recovery Real in Dundee: A review with the Scottish Recovery Network. Scottish Recovery Network. Glasgow: Scottish Recovery Network; 2018.

Crowther A, Taylor A, Toney R, Meddings S, Whale T, Jennings H, et al. The impact of Recovery Colleges on mental health staff, services and society. Epidemiol Psychiatr Sci. 2019;28(5):481–8.

Article   CAS   PubMed   Google Scholar  

Anfossi A. The current state of Recovery Colleges in the UK: final report. Nottingham, UK: Implementing Recovery through Organisational Change (ImROC) and Nottinghamshire Healthcare NHS Foundation Trust; 2017.

Mental Health Commission of Canada. Changing Directions Changing Lives: The Mental Health Strategy for Canada. Calgary, Canada: Mental Health Commission of Canada; 2012.

Giacomini M, Bourgeault I, Dingwall R, De Vries, R. Theory matters in qualitative health research. In: The SAGE Handbook of Qualitative Methods in Health Research. London: SAGE Publications Ltd.; 2010. p. 125–56.

Chapter   Google Scholar  

Tremblay M-C, Martin DH, Macaulay AC, Pluye P. Can we build on social movement theories to develop and improve community-based participatory research? A framework synthesis review. Am J Community Psychol. 2017;59(3-4):333–62.

Maton K. Empowering community settings: agents of individual development, community betterment, and positive social change. Am J Community Psychol. 2008;41:4–21.

Boehnert J. The visual representation of complexity: sixteen key characteristics of complex systems. In: Proceedings of RSD7, Relating Systems Thinking and Design 7; 2018 Oct 23-26; Turin, Italy. Available at http://openresearch.ocadu.ca/id/eprint/2737/

Pierson P. When effect becomes cause: policy feedback and political change. World Politics. 1993;45(4):595–628.

Kingdon JW. Agendas, alternatives and public policies. 2nd ed. New York: Longman; 1995.

Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving research impact through co-creation in community-based health services; literature review and case study. Milbank Q. 2016;94(2):392–429.

Download references

Acknowledgements

We wish to thank all participants in this study for giving their time and for sharing their experiences. We also thank the study’s collaborators who provided important background to the cases contributing to the research design/direction, acted as local gatekeepers to the cases and/or who helped to interpret the data. Over the life of the research project, the collaborators have been: Louise Christie (Scottish Recovery Network), Marion Cooper (CMHA Manitoba & Winnipeg), Olivia Hanley (formerly of the Scottish Community Development Centre), Greg Kyllo (formerly of CMHA National), Erica McDiarmid (formerly of CMHA National), Susan Paxton (Scottish Community Development Centre), Denise Silverstone (CMHA National), Stephanie Skakun (CMHA Manitoba & Winnipeg) and Nicoline Vackerberg (Region Jönköping County). Without their involvement, this study would not have been possible. Finally, we thank Sophie Sarre for her contributions to wave 1 interviewing and early coding framework development and data coding.

This manuscript draws on research supported by the Social Sciences and Humanities Research Council Partnership Development grant no. 890-2018-0116. The funders had no role in the design of the study; in the collection, analysis and interpretation of data; or in writing the manuscript.

Author information

Authors and affiliations.

DeGroote School of Business, McMaster University, 4350 South Service Road, Suite 421, Burlington, ON, L7L 5R8, Canada

Gillian Mulvale & Jenn Green

Florence Nightingale Faculty of Nursing, Midwifery and Palliative Care, King’s College London, London, United Kingdom

Glenn Robert

Institute of Health and Neurodevelopment, Aston University, Birmingham, United Kingdom

Michael Larkin & Shioma-Lei Craythorne

Region Jönköping County, Jönköping, Sweden

Nicoline Vackerberg

The Jönköping Academy for Improvement of Health and Welfare, School of Health and Welfare, Jönköping University, Jönköping, Sweden

Glenn Robert, Nicoline Vackerberg & Sofia Kjellström

Department of Health Research Methods, Evidence, and Impact (HEI), Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada

Puspita Hossain

School of Rehabilitation Science, McMaster University, Hamilton, ON, Canada

Sandra Moll

School of Health and Welfare, Jönköping University, Jönköping, Sweden

SingHealth Office of Regional Health, Singapore Health Services, Singapore, Singapore

You can also search for this author in PubMed   Google Scholar

Contributions

GM, GR, ML, SK and SM conceived of and designed the study. GM, JG, NV, EL and SC collected and analysed the data under the guidance of GM, GR, ML and SK. GM, JG and PH interpreted the data. GM, GR, JG and PH drafted the manuscript. ML, NV, SK, SM, EL and SC reviewed and commented on different versions of the paper. GR, GM and JG revised the manuscript following peer review, in consultation with the other authors. All the authors have read and approved the final manuscript.

Corresponding author

Correspondence to Gillian Mulvale .

Ethics declarations

Ethics approval and consent to participate.

Research ethics clearance was obtained from the relevant academic research ethics boards (McMaster University Research Ethics Board [MREB Project ID 2066], Aston University Ethics Committee [Rec Ref #1611]; King’s College London Research Ethics Office [Reference Number MOD-19/20-17350]; SingHealth Centralised Institutional Review Board [CIRB Ref# 2020/2341]; and Swedish Ethical Review Authority [Etikprövningsmyndigheten, Dnr 2019-06373]), and in light of this, ethics review was waived by the boards of the collaborating organizations (Canadian Mental Health Association, Manitoba & Winnipeg branch, the East of Scotland Research Ethics Service). Participants received letters of information outlining the study objectives, protocol and risks prior to consenting in writing.

Consent for publication

Consent for publication was received from all key informants.

Competing interests

The authors declare that they have no competing interests. All authors approved the final version of the paper.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Provides additional details about the sampling frame (that is, the organizations the interviewees are associated with, the document titles and types).

Additional file 2.

Demonstrates how concepts from the CFIR, DOI, and compatibility gaps frameworks were incorporated into the coding framework.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Mulvale, G., Green, J., Robert, G. et al. Adopting, implementing and assimilating coproduced health and social care innovations involving structurally vulnerable populations: findings from a longitudinal, multiple case study design in Canada, Scotland and Sweden. Health Res Policy Sys 22 , 42 (2024). https://doi.org/10.1186/s12961-024-01130-w

Download citation

Received : 12 November 2023

Accepted : 05 March 2024

Published : 02 April 2024

DOI : https://doi.org/10.1186/s12961-024-01130-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Coproduction
• Structurally vulnerable populations
• Implementation
• Assimilation
• Transformation

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Implement Sci Commun

Developing an implementation research logic model: using a multiple case study design to establish a worked exemplar

Louise czosnek.

1 Mary MacKillop Institute for Health Research, Australian Catholic University, Melbourne, Australia

Eva M. Zopf

2 Cabrini Cancer Institute, The Szalmuk Family Department of Medical Oncology, Cabrini Health, Melbourne, Australia

Prue Cormie

3 Peter MacCallum Cancer Centre, Melbourne, Australia

4 Sir Peter MacCallum Department of Oncology, The University of Melbourne, Melbourne, Australia

Simon Rosenbaum

5 Discipline of Psychiatry and Mental Health, University of New South Wales, Sydney, Australia

6 School of Health Sciences, University of New South Wales, Sydney, Australia

Justin Richards

7 Faculty of Health, Victoria University of Wellington, Wellington, New Zealand

Nicole M. Rankin

8 Faculty of Medicine and Health, University of Sydney, Sydney, Australia

9 Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, Australia

Associated Data

The data that support the findings of this study are available on request from the corresponding author [LC]. The data are not publicly available due to them containing information that could compromise research participant privacy.

Implementation science frameworks explore, interpret, and evaluate different components of the implementation process. By using a program logic approach, implementation frameworks with different purposes can be combined to detail complex interactions. The Implementation Research Logic Model (IRLM) facilitates the development of causal pathways and mechanisms that enable implementation. Critical elements of the IRLM vary across different study designs, and its applicability to synthesizing findings across settings is also under-explored. The dual purpose of this study is to develop an IRLM from an implementation research study that used case study methodology and to demonstrate the utility of the IRLM to synthesize findings across case sites.

The method used in the exemplar project and the alignment of the IRLM to case study methodology are described. Cases were purposely selected using replication logic and represent organizations that have embedded exercise in routine care for people with cancer or mental illness. Four data sources were selected: semi-structured interviews with purposely selected staff, organizational document review, observations, and a survey using the Program Sustainability Assessment Tool (PSAT). Framework analysis was used, and an IRLM was produced at each case site. Similar elements within the individual IRLM were identified, extracted, and re-produced to synthesize findings across sites and represent the generalized, cross-case findings.

The IRLM was embedded within multiple stages of the study, including data collection, analysis, and reporting transparency. Between 33-44 determinants and 36-44 implementation strategies were identified at sites that informed individual IRLMs. An example of generalized findings describing “intervention adaptability” demonstrated similarities in determinant detail and mechanisms of implementation strategies across sites. However, different strategies were applied to address similar determinants. Dependent and bi-directional relationships operated along the causal pathway that influenced implementation outcomes.

Conclusions

Case study methods help address implementation research priorities, including developing causal pathways and mechanisms. Embedding the IRLM within the case study approach provided structure and added to the transparency and replicability of the study. Identifying the similar elements across sites helped synthesize findings and give a general explanation of the implementation process. Detailing the methods provides an example for replication that can build generalizable knowledge in implementation research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00337-8.

Contributions to the literature

• Logic models can help understand how and why evidence-based interventions (EBIs) work to produce intended outcomes.
• The implementation research logic model (IRLM) provides a method to understand causal pathways, including determinants, implementation strategies, mechanisms, and implementation outcomes.
• We describe an exemplar project using a multiple case study design that embeds the IRLM at multiple stages. The exemplar explains how the IRLM helped synthesize findings across sites by identifying the common elements within the causal pathway.
• By detailing the exemplar methods, we offer insights into how this approach of using the IRLM is generalizable and can be replicated in other studies.

The practice of implementation aims to get “someone…, somewhere… to do something differently” [ 1 ]. Typically, this involves changing individual behaviors and organizational processes to improve the use of evidence-based interventions (EBIs). To understand this change, implementation science applies different theories, models, and frameworks (hereafter “frameworks”) to describe and evaluate the factors and steps in the implementation process [ 2 – 5 ]. Implementation science provides much-needed theoretical frameworks and a structured approach to process evaluations. One or more frameworks are often used within a program of work to investigate the different stages and elements of implementation [ 6 ]. Researchers have acknowledged that the dynamic implementation process could benefit from using logic models [ 7 ]. Logic models offer a systematic approach to combining multiple frameworks and to building causal pathways that explain the mechanisms behind individual and organizational change.

Logic models visually represent how an EBI is intended to work [ 8 ]. They link the available resources with the activities undertaken, the immediate outputs of this work, and the intermediate outcomes and longer-term impacts [ 8 , 9 ]. Through this process, causal pathways are identified. For implementation research, the causal pathway provides the interconnection between a chosen EBI, determinants, implementation strategies, and implementation outcomes [ 10 ]. Testing causal mechanisms in the research translation pathway will likely dominate the next wave of implementation research [ 11 , 12 ]. Causal mechanisms (or mechanisms of change) are the “process or event through which an implementation strategy operates to affect desired implementation outcomes” [ 13 ]. Identifying mechanisms can improve implementation strategies’ selection, prioritization, and targeting [ 12 , 13 ]. This provides an efficient and evidence-informed approach to implementation.

Implementation researchers have proposed several methods to develop and examine causal pathways [ 14 , 15 ] and mechanisms [ 16 , 17 ]. This includes formalizing the inherent relationship between frameworks via developing the Implementation Research Logic Model (IRLM) [ 7 ]. The IRLM is a logic model designed to improve the rigor and reproducibility of implementation research. It specifies the relationship between elements of implementation (determinant, strategies, and outcomes) and the mechanisms of change. To do this, it recommends linking implementation frameworks or relevant taxonomies (e.g., determinant and evaluation frameworks and implementation strategy taxonomy). The IRLM authors suggest the tool has multiple uses, including planning, executing, and reporting on the implementation process and synthesizing implementation findings across different contexts [ 7 ]. During its development, the IRLM was tested to confirm its utility in planning, executing, and reporting; however, its utility in synthesizing findings across different contexts is ongoing. Users of the tool are encouraged to consider three principles: (1) comprehensiveness in reporting determinants, implementation strategies, and implementation outcomes; (2) specifying the conceptual relationships via diagrammatic tools such as colors and arrows; and (3) detailing important elements of the study design. Further, the authors also recognize that critical elements of IRLM will vary across different study designs.

This study describes the development of an IRLM from a multiple case study design. Case study methodology can answer “how and why” questions about implementation. They enable researchers to develop a rich, in-depth understanding of a contemporary phenomenon within its natural context [ 18 – 21 ]. These methods can create coherence in the dynamic context in which EBIs exist [ 22 , 23 ]. Case studies are common in implementation research [ 24 – 30 ], with multiple case study designs suitable for undertaking comparisons across contexts [ 31 , 32 ]. However, they are infrequently applied to establish mechanisms [ 11 ] or combine implementation elements to synthesize findings across contexts (as possible through the IRLM). Hollick and colleagues [ 33 ] undertook a comparative case study, guided by a determinant framework, to explore how context influences successful implementation. The authors contrasted determinants across sites where implementation was successful versus sites where implementation failed. The study did not extend to identifying implementation strategies or mechanisms. By contrast, van Zelm et al. [ 31 ] undertook a theory-driven evaluation of successful implementation across ten hospitals. They used joint displays to present mechanisms of change aligned with evaluation outcomes; however, they did not identify the implementation strategies within the causal pathway. Our study seeks to build on these works and explore the utility of the IRLM in synthesizing findings across sites. The dual objectives of this paper were to:

• Describe how case study methods can be applied to develop an IRLM
• Demonstrate the utility of the IRLM in synthesizing implementation findings across case sites.

In this section, we describe the methods used in the exemplar case study and the alignment of the IRLM to this approach. The exemplar study explored the implementation of exercise EBIs in the context of the Australian healthcare system. The exemplar study aimed to investigate the integration of exercise EBIs within routine mental illness or cancer care. The evidence base detailing the therapeutic benefits of exercise for non-communicable diseases such as cancer and mental illness are extensively documented [ 34 – 36 ] but inconsistently implemented as part of routine care [ 37 – 44 ].

Additional file 1 provides the Standards for Reporting Qualitative Research (SRQR).

Case study approach

We adopted an approach to case studies based on the methods described by Yin [ 18 ]. This approach is said to have post-positivist philosophical leanings, which are typically associated with the quantitative paradigm [ 19 , 45 , 46 ]. This is evidenced by the structured, deductive approach to the methods that are described with a constant lens on objectivity, validity, and generalization [ 46 ]. Yin’s approach to case studies aligns with the IRLM for several reasons. The IRLM is designed to use established implementation frameworks. The two frameworks and one taxonomy applied in our exemplar were the Consolidated Framework for Implementation Research (CFIR) [ 47 ], Expert Recommendations for Implementing Change (ERIC) [ 48 ], and Proctor et al.’s implementation outcomes framework [ 49 ]. These frameworks guided multiple aspects of our study (see Table ​ Table1). 1 ). Commencing an implementation study with a preconceived plan based upon established frameworks is deductive [ 22 ]. Second, the IRLM has its foundation in logic modeling to develop cause and effect relationships [ 8 ]. Yin advocates using logic models to analyze case study findings [ 18 ]. They argue that developing logic models encourages researchers to iterate and consider plausible counterfactual explanations before upholding the causal pathway. Further, Yin notes that case studies are particularly valuable for explaining the transitions and context within the cause-and-effect relationship [ 18 ]. In our exemplar, the transition was the mechanism between the implementation strategy and implementation outcome. Finally, the proposed function of IRLM to synthesize findings across sites aligns with the exemplar study that used a multiple case approach. Multiple case studies aim to develop generalizable knowledge [ 18 , 50 ].

Theoretical application within the study and operational definitions/measures for implementation outcomes

EBI evidence-based intervention, PSAT Program Sustainability Assessment Tool, CFIR Consolidated Framework for Implementation Research, ERIC Expert Recommendations for Implementing Change

Case study selection and boundaries

A unique feature of Yin’s approach to multiple case studies is using replication logic to select cases [ 18 ]. Cases are chosen to demonstrate similarities (literal replication) or differences for anticipated reasons (theoretical replication) [ 18 ]. In the exemplar study, the cases were purposely selected using literal replication and displayed several common characteristics. First, all cases had delivered exercise EBIs within normal operations for at least 12 months. Second, each case site delivered exercise EBIs as part of routine care for a non-communicable disease (cancer or mental illness diagnosis). Finally, each site delivered the exercise EBI within the existing governance structures of the Australian healthcare system. That is, the organizations used established funding and service delivery models of the Australian healthcare system.

Using replication logic, we posited that sites would exhibit some similarities in the implementation process across contexts (literal replication). However, based on existing implementation literature [ 32 , 51 – 53 ], we expected sites to adapt the EBIs through the implementation process. The determinant analysis should explain these adaptions, which is informed by the CFIR (theoretical replication). Finally, in case study methods, clearly defining the boundaries of each case and the units of analysis, such as individual, the organization or intervention, helps focus the research. We considered each healthcare organization as a separate case. Within that, organizational-level analysis [ 18 , 54 ] and operationalizing the implementation outcomes focused inquiry (Table ​ (Table1 1 ).

Data collection

During the study conceptualization for the exemplar, we mapped the data sources to the different elements of the IRLM (Fig. ​ (Fig.1). 1 ). Four primary data sources informed data collection: (1) semi-structured interviews with staff; (2) document review (such as meeting minutes, strategic plans, and consultant reports); (3) naturalistic observations; and (4) a validated survey (Program Sustainability Assessment Tool (PSAT)). A case study database was developed using Microsoft Excel to manage and organize data collection [ 18 , 54 ].

Conceptual frame for the study

Semi-structured interviews

An interview guide was developed, informed by the CFIR interview guide tool [ 55 ]. Questions were selected across the five domains of the CFIR, which aligned with the delineation of determinant domains in the IRLM. Purposeful selection was used to identify staff for the interviews [ 56 ]. Adequate sample size in qualitative studies, particularly regarding the number of interviews, is often determined when data saturation is reached [ 57 , 58 ]. Unfortunately, there is little consensus on the definition of saturation [ 59 ], how to interpret when it has occurred [ 57 ], or whether it is possible to pre-determine in qualitative studies [ 60 ]. The number of participants in this study was determined based on the staff’s differential experience with the exercise EBI and their role in the organization. This approach sought to obtain a rounded view of how the EBI operated at each site [ 23 , 61 ]. Focusing on staff experiences also aligned with the organizational lens that bounded the study. Typical roles identified for the semi-structured interviews included the health professional delivering the EBI, the program manager responsible for the EBI, an organizational executive, referral sources, and other health professionals (e.g., nurses, allied health). Between five and ten interviews were conducted at each site. Interview times ranged from 16 to 72 min, most lasting around 40 min per participant.

Document review

A checklist informed by case study literature was developed outlining the typical documents the research team was seeking [ 18 ]. The types of documents sought to review included job descriptions, strategic plans/planning documents, operating procedures and organizational policies, communications (e.g., website, media releases, email, meeting minutes), annual reports, administrative databases/files, evaluation reports, third party consultant reports, and routinely collected numerical data that measured implementation outcomes [ 27 ]. As each document was identified, it was numbered, dated, and recorded in the case study database with a short description of the content related to the research aims and the corresponding IRLM construct. Between 24 and 33 documents were accessed at each site. A total of 116 documents were reviewed across the case sites.

Naturalistic observations

The onsite observations occurred over 1 week, wherein typical organizational operations were viewed. The research team interacted with staff, asked questions, and sought clarification of what was being observed; however, they did not disrupt the usual work routines. Observations allowed us to understand how the exercise EBI operated and contrast that with documented processes and procedures. They also provided the opportunity to observe non-verbal cues and interactions between staff. While onsite, case notes were recorded directly into the case study database [ 62 , 63 ]. Between 15 and 40 h were spent on observations per site. A total of 95 h was spent across sites on direct observations.

Program sustainability assessment tool (survey)

The PSAT is a planning and evaluation tool that assesses the sustainability of an intervention across eight domains [ 64 – 66 ]: (1) environmental support, (2) funding stability, (3) partnerships, (4) organizational capacity, (5) program evaluation, (6) program adaption, (7) communication, and (8) strategic planning [ 64 , 65 ]. The PSAT was administered to a subset of at least three participants per site who completed the semi-structured interview. The results were then pooled to provide an organization-wide view of EBI sustainability. Three participants per case site are consistent with previous studies that have used the tool [ 67 , 68 ] and recommendations for appropriate use [ 65 , 69 ].

We included a validated measure of sustainability, recognizing calls to improve understanding of this aspect of implementation [ 70 – 72 ]. Noting the limited number of measurement tools for evaluating sustainability [ 73 ], the PSAT’s characteristics displayed the best alignment with the study aims. To determine “best alignment,” we deferred to a study by Lennox and colleagues that helps researchers select suitable measurement tools based on the conceptualization of sustainability in the study [ 71 ]. The PSAT provides a multi-level view of sustainability. It is a measurement tool that can be triangulated with other implementation frameworks, such as the CFIR [ 74 ], to interrogate better and understand the later stages of implementation. Further, the tool provides a contemporary account of an EBIs capacity for sustainability [ 75 ]. This is consistent with case study methods, which explore complex, contemporary, real-life phenomena.

The voluminous data collection that is possible through case studies, and is often viewed as a challenge of the method [ 19 ], was advantageous to developing the IRLM in the exemplar and identifying the causal pathways. First, it aided three types of triangulation through the study (method, theory, and data source triangulation) [ 76 ]. Method triangulation involved collecting evidence via four methods: interview, observations, document review, and survey. Theoretical triangulation involved applying two frameworks and one taxonomy to understand and interpret the findings. Data source triangulation involved selecting participants with different roles within the organization to gain multiple perspectives about the phenomena being studied. Second, data collection facilitated depth and nuance in detailing determinants and implementation strategies. For the determinant analysis, this illuminated the subtleties within context and improved confidence and accuracy for prioritizing determinants. As case studies are essentially “naturalistic” studies, they provide insight into strategies that are implementable in pragmatic settings. Finally, the design’s flexibility enabled the integration of a survey and routinely collected numerical data as evaluation measures for implementation outcomes. This allowed us to contrast “numbers” against participants’ subjective experience of implementation [ 77 ].

Data analysis

Descriptive statistics were calculated for the PSAT and combined with the three other data sources wherein framework analysis [ 78 , 79 ] was used to analyze the data. Framework analysis includes five main phases: familiarization, identifying a thematic framework, indexing, charting, and mapping and interpretation [ 78 ]. Familiarization occurred concurrently with data collection, and the thematic frame was aligned to the two frameworks and one taxonomy we applied to the IRLM. To index and chart the data, the raw data was uploaded into NVivo 12 [ 80 ]. Codes were established to guide indexing that aligned with the thematic frame. That is, determinants within the CFIR [ 47 ], implementation strategies listed in ERIC [ 48 ], and the implementation outcomes [ 49 ] of acceptability, fidelity, penetration, and sustainability were used as codes in NVivo 12. This process produced a framework matrix that summarized the information housed under each code at each case site.

The final step of framework analysis involves mapping and interpreting the data. We used the IRLM to map and interpret the data in the exemplar. First, we identified the core elements of the implemented exercise EBI. Next, we applied the CFIR valance and strength coding to prioritize the contextual determinants. Then, we identified the implementation strategies used to address the contextual determinants. Finally, we provided a rationale (a causal mechanism) for how these strategies worked to address barriers and contribute to specific implementation outcomes. The systematic approach advocated by the IRLM provided a transparent representation of the causal pathway underpinning the implementation of the exercise EBIs. This process was followed at each case site to produce an IRLM for each organization. To compare, contrast, and synthesize findings across sites, we identified the similarities and differences in the individual IRLMs and then developed an IRLM that explained a generalized process for implementation. Through the development of the causal pathway and mechanisms, we deferred to existing literature seeking to establish these relationships [ 81 – 83 ]. Aligned with case study methods, this facilitated an iterative process of constant comparison and challenging the proposed causal relationships. Smith and colleagues advise that the IRLM “might be viewed as a somewhat simplified format,” and users are encouraged to “iterate on the design of the IRLM to increase its utility” [ 7 ]. Thus, we re-designed the IRLM within a traditional logic model structure to help make sense of the data collected through the case studies. Figure ​ Figure1 1 depicts the conceptual frame for the study and provides a graphical representation of how the IRLM pathway was produced.

The results are presented with reference to the three principles of the IRLM: comprehensiveness, indicating the key conceptual relationship and specifying critical study design . The case study method allowed for comprehensiveness through the data collection and analysis described above. The mean number of data sources informing the analysis and development of the causal pathway at each case site was 63.75 (interviews ( M = 7), observational hours ( M =23.75), PSAT ( M =4), and document review ( M = 29). This resulted in more than 30 determinants and a similar number of implementation strategies identified at each site (determinant range per site = 33–44; implementation strategy range per site = 36–44). Developing a framework matrix meant that each determinant (prioritized and other), implementation strategy, and implementation outcome were captured. The matrix provided a direct link to the data sources that informed the content within each construct. An example from each construct was collated alongside the summary to evidence the findings.

The key conceptual relationship was articulated in a traditional linear process by aligning determinant → implementation strategy → mechanism → implementation outcome, as per the IRLM. To synthesize findings across sites, we compared and contrasted the results within each of the individual IRLM and extracted similar elements to develop a generalized IRLM that represents cross-case findings. By redeveloping the IRLM within a traditional logic model structure, we added visual representations of the bi-directional and dependent relationships, illuminating the dynamism within the implementation process. To illustrate, intervention adaptability was a prioritized determinant and enabler across sites. Healthcare providers recognized that adapting and tailoring exercise EBIs increased “fit” with consumer needs. This also extended to adapting how healthcare providers referred consumers to exercise so that it was easy in the context of their other work priorities. Successful adaption was contingent upon a qualified workforce with the required skills and competencies to enact change. Different implementation strategies were used to make adaptions across sites, such as promoting adaptability and using data experts. However, despite the different strategies, successful adaptation created positive bi-directional relationships. That is, healthcare providers’ confidence and trust in the EBI grew as consumer engagement increased and clinical improvements were observed. This triggered greater engagement with the EBI (e.g., acceptability → penetration → sustainability), albeit the degree of engagement differed across sites. Figure ​ Figure2 2 illustrates this relationship within the IRLM and provides a contrasting relationship by highlighting how a prioritized barrier across sites (available resources) was addressed.

Example of intervention adaptability (E) contrasted with available resources (B) within a synthesised IRLM across case sites

The final principle is to specify critical study design , wherein we have described how case study methodology was used to develop the IRLM exemplar. Our intention was to produce an explanatory causal pathway for the implementation process. The implementation outcomes of acceptability and fidelity were measured at the level of the provider, and penetration and sustainability were measured at the organizational level [ 49 ]. Service level and clinical level outcomes were not identified for a priori measurement throughout the study. We did identify evidence of clinical outcomes that supported our overall findings via the document review. Historical evaluations on the service indicated patients increased their exercise level or demonstrated a change in symptomology/function. The implementation strategies specified in the study were those chosen by the organizations. We did not attempt to augment routine practice or change implementation outcomes by introducing new strategies. The barriers across sites were represented with a (B) symbol and enablers with an (E) symbol in the IRLM. In the individual IRLM, consistent determinants and strategies were highlighted (via bolding) to support extraction. Finally, within the generalized IRLM, the implementation strategies are grouped according to the ERIC taxonomy category. This accounts for the different strategies applied to achieve similar outcomes across case studies.

This study provides a comprehensive overview that uses case study methodology to develop an IRLM in an implementation research project. Using an exemplar that examines implementation in different healthcare settings, we illustrate how the IRLM (that documents the causal pathways and mechanisms) was developed and enabled the synthesis of findings across sites.

Case study methodologies are fraught with inconsistencies in terminology and approach. We adopted the method described by Yin. Its guiding paradigm, which is rooted in objectivity, means it can be viewed as less flexible than other approaches [ 46 , 84 ]. We found the approach offered sufficient flexibility within the frame of a defined process. We argue that the defined process adds to the rigor and reproducibility of the study, which is consistent with the principles of implementation science. That is, accessing multiple sources of evidence, applying replication logic to select cases, maintaining a case study database, and developing logic models to establish causal pathways, demonstrates the reliability and validity of the study. The method was flexible enough to embed the IRLM within multiple phases of the study design, including conceptualization, philosophical alignment, and analysis. Paparini and colleagues [ 85 ] are developing guidance that recognizes the challenges and unmet value of case study methods for implementation research. This work, supported by the UK Medical Research Council, aims to enhance the conceptualization, application, analysis, and reporting of case studies. This should encourage and support researchers to use case study methods in implementation research with increased confidence.

The IRLM produced a relatively linear depiction of the relationship between context, strategies, and outcomes in our exemplar. However, as noted by the authors of the IRLM, the implementation process is rarely linear. If the tool is applied too rigidly, it may inadvertently depict an overly simplistic view of a complex process. To address this, we redeveloped the IRLM within a traditional logic model structure, adding visual representations of the dependent and bidirectional relationships evident within the general IRLM pathway [ 86 ]. Further, developing a general IRLM of cross-case findings that synthesized results involved a more inductive approach to identifying and extracting similar elements. It required the research team to consider broader patterns in the data before offering a prospective account of the implementation process. This was in contrast to the earlier analysis phases that directly mapped determinants and strategies to the CFIR and ERIC taxonomy. We argue that extracting similar elements is analogous to approaches that have variously been described as portable elements [ 87 ], common elements [ 88 ], or generalization by mechanism [ 89 ]. While defined and approached slightly differently, these approaches aim to identify elements frequently shared across effective EBIs and thus can form the basis of future EBIs to increase their utility, efficiency, and effectiveness [ 88 ]. We identified similarities related to determinant detail and mechanism of different implementation strategies across sites. This finding supports the view that many implementation strategies could be suitable, and selecting the “right mix” is challenging [ 16 ]. Identifying common mechanisms, such as increased motivation, skill acquisition, or optimizing workflow, enabled elucidation of the important functions of strategies. This can help inform the selection of appropriate strategies in future implementation efforts.

Finally, by developing individual IRLMs and then re-producing a general IRLM, we synthesized findings across sites and offered generalized findings. The ability to generalize from case studies is debated [ 89 , 90 ], with some considering the concept a fallacy [ 91 ]. That is, the purpose of qualitative research is to develop a richness through data that is situated within a unique context. Trying to extrapolate from findings is at odds with exploring unique context. We suggest the method described herein and the application of IRLM could be best applied to a form of generalization called ‘transferability’ [ 91 , 92 ]. This suggests that findings from one study can be transferred to another setting or population group. In this approach, the new site takes the information supplied and determines those aspects that would fit with their unique environment. We argue that elucidating the implementation process across multiple sites improves the confidence with which certain “elements” could be applied to future implementation efforts. For example, our approach may also be helpful for multi-site implementation studies that use methods other than case studies. Developing a general IRLM through study conceptualization could identify consistencies in baseline implementation status across sites. Multi-site implementation projects may seek to introduce and empirically test implementation strategies, such as via a cluster randomized controlled trial [ 93 ]. Within this study design, baseline comparison between control and intervention sites might extend to a comparison of organizational type, location and size, and individual characteristics, but not the chosen implementation strategies [ 94 ]. Applying the approach described within our study could enhance our understanding of how to support effective implementation.

Limitations

After the research team conceived this study, the authors of the PSAT validated another tool for use in clinical settings (Clinical Sustainability Assessment Tool (CSAT)) [ 95 ]. This tool appears to align better with our study design due to its explicit focus on maintaining structured clinical care practices. The use of multiple data sources and consistency in some elements across the PSAT and CSAT should minimize the limitations in using the PSAT survey tool. At most case sites, limited staff were involved in developing and implementing exercise EBI. Participants who self-selected for interviews may be more invested in assuring positive outcomes for the exercise EBI. Inviting participants from various roles was intended to reduce selection bias. Finally, we recognize recent correspondence suggesting the IRLM misses a critical step in the causal pathway. That is the mechanism between determinant and selection of an appropriate implementation strategy [ 96 ]. Similarly, Lewis and colleagues note that additional elements, including pre-conditions, moderators, and mediators (distal and proximal), exist within the causal pathway [ 13 ]. Through the iterative process of developing the IRLM, decisions were made about the determinant → implementation strategy relationship; however, this is not captured in the IRLM. Secondary analysis of the case study data would allow elucidation of these relationships, as this information can be extracted through the case study database. This was outside the scope of the exemplar study.

Developing an IRLM via case study methods proved useful in identifying causal pathways and mechanisms. The IRLM can complement and enhance the study design by providing a consistent and structured approach. In detailing our approach, we offer an example of how multiple case study designs that embed the IRLM can aid the synthesis of findings across sites. It also provides a method that can be replicated in future studies. Such transparency adds to the quality, reliability, and validity of implementation research.

Acknowledgements

The authors would like to acknowledge the healthcare organizations and staff who supported the study.

Authors’ contributions

LC, EZ, SR, JR, PC, and NR contributed to the conceptualization of the study. LC undertook the data collection, and LC, EZ, SR, JR, PC, and NR supported the analysis. The first draft of the manuscript was written by LC with NR and EZ providing first review. LC, EZ, SR, JR, PC, and NR commented on previous versions of the manuscript and provided critical review. All authors read and approved the final manuscript.

SR is funded by an NHMRC Early Career Fellowship (APP1123336). The funding body had no role in the study design, data collection, data analysis, interpretation, or manuscript development.

Availability of data and materials

Declarations.

This study is approved by Sydney Local Health District Human Research Ethics Committee - Concord Repatriation General Hospital (2019/ {"type":"entrez-protein","attrs":{"text":"ETH11806","term_id":"565702355"}} ETH11806 ). Ethical approval is also supplied by Australian Catholic University (2018-279E), Peter MacCallum Cancer Centre (19/175), North Sydney Local Health District - Macquarie Hospital (2019/ {"type":"entrez-protein","attrs":{"text":"STE14595","term_id":"1438690294"}} STE14595 ), and Alfred Health (516-19).

Not applicable.

PC is the recipient of a Victorian Government Mid-Career Research Fellowship through the Victorian Cancer Agency. PC is the Founder and Director of EX-MED Cancer Ltd, a not-for-profit organization that provides exercise medicine services to people with cancer. PC is the Director of Exercise Oncology EDU Pty Ltd, a company that provides fee for service training courses to upskill exercise professionals in delivering exercise to people with cancer.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.