The Advantages and Limitations of Single Case Study Analysis

limitations of a qualitative case study

As Andrew Bennett and Colin Elman have recently noted, qualitative research methods presently enjoy “an almost unprecedented popularity and vitality… in the international relations sub-field”, such that they are now “indisputably prominent, if not pre-eminent” (2010: 499). This is, they suggest, due in no small part to the considerable advantages that case study methods in particular have to offer in studying the “complex and relatively unstructured and infrequent phenomena that lie at the heart of the subfield” (Bennett and Elman, 2007: 171). Using selected examples from within the International Relations literature[1], this paper aims to provide a brief overview of the main principles and distinctive advantages and limitations of single case study analysis. Divided into three inter-related sections, the paper therefore begins by first identifying the underlying principles that serve to constitute the case study as a particular research strategy, noting the somewhat contested nature of the approach in ontological, epistemological, and methodological terms. The second part then looks to the principal single case study types and their associated advantages, including those from within the recent ‘third generation’ of qualitative International Relations (IR) research. The final section of the paper then discusses the most commonly articulated limitations of single case studies; while accepting their susceptibility to criticism, it is however suggested that such weaknesses are somewhat exaggerated. The paper concludes that single case study analysis has a great deal to offer as a means of both understanding and explaining contemporary international relations.

The term ‘case study’, John Gerring has suggested, is “a definitional morass… Evidently, researchers have many different things in mind when they talk about case study research” (2006a: 17). It is possible, however, to distil some of the more commonly-agreed principles. One of the most prominent advocates of case study research, Robert Yin (2009: 14) defines it as “an empirical enquiry that investigates a contemporary phenomenon in depth and within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident”. What this definition usefully captures is that case studies are intended – unlike more superficial and generalising methods – to provide a level of detail and understanding, similar to the ethnographer Clifford Geertz’s (1973) notion of ‘thick description’, that allows for the thorough analysis of the complex and particularistic nature of distinct phenomena. Another frequently cited proponent of the approach, Robert Stake, notes that as a form of research the case study “is defined by interest in an individual case, not by the methods of inquiry used”, and that “the object of study is a specific, unique, bounded system” (2008: 443, 445). As such, three key points can be derived from this – respectively concerning issues of ontology, epistemology, and methodology – that are central to the principles of single case study research.

First, the vital notion of ‘boundedness’ when it comes to the particular unit of analysis means that defining principles should incorporate both the synchronic (spatial) and diachronic (temporal) elements of any so-called ‘case’. As Gerring puts it, a case study should be “an intensive study of a single unit… a spatially bounded phenomenon – e.g. a nation-state, revolution, political party, election, or person – observed at a single point in time or over some delimited period of time” (2004: 342). It is important to note, however, that – whereas Gerring refers to a single unit of analysis – it may be that attention also necessarily be given to particular sub-units. This points to the important difference between what Yin refers to as an ‘holistic’ case design, with a single unit of analysis, and an ’embedded’ case design with multiple units of analysis (Yin, 2009: 50-52). The former, for example, would examine only the overall nature of an international organization, whereas the latter would also look to specific departments, programmes, or policies etc.

Secondly, as Tim May notes of the case study approach, “even the most fervent advocates acknowledge that the term has entered into understandings with little specification or discussion of purpose and process” (2011: 220). One of the principal reasons for this, he argues, is the relationship between the use of case studies in social research and the differing epistemological traditions – positivist, interpretivist, and others – within which it has been utilised. Philosophy of science concerns are obviously a complex issue, and beyond the scope of much of this paper. That said, the issue of how it is that we know what we know – of whether or not a single independent reality exists of which we as researchers can seek to provide explanation – does lead us to an important distinction to be made between so-called idiographic and nomothetic case studies (Gerring, 2006b). The former refers to those which purport to explain only a single case, are concerned with particularisation, and hence are typically (although not exclusively) associated with more interpretivist approaches. The latter are those focused studies that reflect upon a larger population and are more concerned with generalisation, as is often so with more positivist approaches[2]. The importance of this distinction, and its relation to the advantages and limitations of single case study analysis, is returned to below.

Thirdly, in methodological terms, given that the case study has often been seen as more of an interpretivist and idiographic tool, it has also been associated with a distinctly qualitative approach (Bryman, 2009: 67-68). However, as Yin notes, case studies can – like all forms of social science research – be exploratory, descriptive, and/or explanatory in nature. It is “a common misconception”, he notes, “that the various research methods should be arrayed hierarchically… many social scientists still deeply believe that case studies are only appropriate for the exploratory phase of an investigation” (Yin, 2009: 6). If case studies can reliably perform any or all three of these roles – and given that their in-depth approach may also require multiple sources of data and the within-case triangulation of methods – then it becomes readily apparent that they should not be limited to only one research paradigm. Exploratory and descriptive studies usually tend toward the qualitative and inductive, whereas explanatory studies are more often quantitative and deductive (David and Sutton, 2011: 165-166). As such, the association of case study analysis with a qualitative approach is a “methodological affinity, not a definitional requirement” (Gerring, 2006a: 36). It is perhaps better to think of case studies as transparadigmatic; it is mistaken to assume single case study analysis to adhere exclusively to a qualitative methodology (or an interpretivist epistemology) even if it – or rather, practitioners of it – may be so inclined. By extension, this also implies that single case study analysis therefore remains an option for a multitude of IR theories and issue areas; it is how this can be put to researchers’ advantage that is the subject of the next section.

Having elucidated the defining principles of the single case study approach, the paper now turns to an overview of its main benefits. As noted above, a lack of consensus still exists within the wider social science literature on the principles and purposes – and by extension the advantages and limitations – of case study research. Given that this paper is directed towards the particular sub-field of International Relations, it suggests Bennett and Elman’s (2010) more discipline-specific understanding of contemporary case study methods as an analytical framework. It begins however, by discussing Harry Eckstein’s seminal (1975) contribution to the potential advantages of the case study approach within the wider social sciences.

Eckstein proposed a taxonomy which usefully identified what he considered to be the five most relevant types of case study. Firstly were so-called configurative-idiographic studies, distinctly interpretivist in orientation and predicated on the assumption that “one cannot attain prediction and control in the natural science sense, but only understanding ( verstehen )… subjective values and modes of cognition are crucial” (1975: 132). Eckstein’s own sceptical view was that any interpreter ‘simply’ considers a body of observations that are not self-explanatory and “without hard rules of interpretation, may discern in them any number of patterns that are more or less equally plausible” (1975: 134). Those of a more post-modernist bent, of course – sharing an “incredulity towards meta-narratives”, in Lyotard’s (1994: xxiv) evocative phrase – would instead suggest that this more free-form approach actually be advantageous in delving into the subtleties and particularities of individual cases.

Eckstein’s four other types of case study, meanwhile, promote a more nomothetic (and positivist) usage. As described, disciplined-configurative studies were essentially about the use of pre-existing general theories, with a case acting “passively, in the main, as a receptacle for putting theories to work” (Eckstein, 1975: 136). As opposed to the opportunity this presented primarily for theory application, Eckstein identified heuristic case studies as explicit theoretical stimulants – thus having instead the intended advantage of theory-building. So-called p lausibility probes entailed preliminary attempts to determine whether initial hypotheses should be considered sound enough to warrant more rigorous and extensive testing. Finally, and perhaps most notably, Eckstein then outlined the idea of crucial case studies , within which he also included the idea of ‘most-likely’ and ‘least-likely’ cases; the essential characteristic of crucial cases being their specific theory-testing function.

Whilst Eckstein’s was an early contribution to refining the case study approach, Yin’s (2009: 47-52) more recent delineation of possible single case designs similarly assigns them roles in the applying, testing, or building of theory, as well as in the study of unique cases[3]. As a subset of the latter, however, Jack Levy (2008) notes that the advantages of idiographic cases are actually twofold. Firstly, as inductive/descriptive cases – akin to Eckstein’s configurative-idiographic cases – whereby they are highly descriptive, lacking in an explicit theoretical framework and therefore taking the form of “total history”. Secondly, they can operate as theory-guided case studies, but ones that seek only to explain or interpret a single historical episode rather than generalise beyond the case. Not only does this therefore incorporate ‘single-outcome’ studies concerned with establishing causal inference (Gerring, 2006b), it also provides room for the more postmodern approaches within IR theory, such as discourse analysis, that may have developed a distinct methodology but do not seek traditional social scientific forms of explanation.

Applying specifically to the state of the field in contemporary IR, Bennett and Elman identify a ‘third generation’ of mainstream qualitative scholars – rooted in a pragmatic scientific realist epistemology and advocating a pluralistic approach to methodology – that have, over the last fifteen years, “revised or added to essentially every aspect of traditional case study research methods” (2010: 502). They identify ‘process tracing’ as having emerged from this as a central method of within-case analysis. As Bennett and Checkel observe, this carries the advantage of offering a methodologically rigorous “analysis of evidence on processes, sequences, and conjunctures of events within a case, for the purposes of either developing or testing hypotheses about causal mechanisms that might causally explain the case” (2012: 10).

Harnessing various methods, process tracing may entail the inductive use of evidence from within a case to develop explanatory hypotheses, and deductive examination of the observable implications of hypothesised causal mechanisms to test their explanatory capability[4]. It involves providing not only a coherent explanation of the key sequential steps in a hypothesised process, but also sensitivity to alternative explanations as well as potential biases in the available evidence (Bennett and Elman 2010: 503-504). John Owen (1994), for example, demonstrates the advantages of process tracing in analysing whether the causal factors underpinning democratic peace theory are – as liberalism suggests – not epiphenomenal, but variously normative, institutional, or some given combination of the two or other unexplained mechanism inherent to liberal states. Within-case process tracing has also been identified as advantageous in addressing the complexity of path-dependent explanations and critical junctures – as for example with the development of political regime types – and their constituent elements of causal possibility, contingency, closure, and constraint (Bennett and Elman, 2006b).

Bennett and Elman (2010: 505-506) also identify the advantages of single case studies that are implicitly comparative: deviant, most-likely, least-likely, and crucial cases. Of these, so-called deviant cases are those whose outcome does not fit with prior theoretical expectations or wider empirical patterns – again, the use of inductive process tracing has the advantage of potentially generating new hypotheses from these, either particular to that individual case or potentially generalisable to a broader population. A classic example here is that of post-independence India as an outlier to the standard modernisation theory of democratisation, which holds that higher levels of socio-economic development are typically required for the transition to, and consolidation of, democratic rule (Lipset, 1959; Diamond, 1992). Absent these factors, MacMillan’s single case study analysis (2008) suggests the particularistic importance of the British colonial heritage, the ideology and leadership of the Indian National Congress, and the size and heterogeneity of the federal state.

Most-likely cases, as per Eckstein above, are those in which a theory is to be considered likely to provide a good explanation if it is to have any application at all, whereas least-likely cases are ‘tough test’ ones in which the posited theory is unlikely to provide good explanation (Bennett and Elman, 2010: 505). Levy (2008) neatly refers to the inferential logic of the least-likely case as the ‘Sinatra inference’ – if a theory can make it here, it can make it anywhere. Conversely, if a theory cannot pass a most-likely case, it is seriously impugned. Single case analysis can therefore be valuable for the testing of theoretical propositions, provided that predictions are relatively precise and measurement error is low (Levy, 2008: 12-13). As Gerring rightly observes of this potential for falsification:

“a positivist orientation toward the work of social science militates toward a greater appreciation of the case study format, not a denigration of that format, as is usually supposed” (Gerring, 2007: 247, emphasis added).

In summary, the various forms of single case study analysis can – through the application of multiple qualitative and/or quantitative research methods – provide a nuanced, empirically-rich, holistic account of specific phenomena. This may be particularly appropriate for those phenomena that are simply less amenable to more superficial measures and tests (or indeed any substantive form of quantification) as well as those for which our reasons for understanding and/or explaining them are irreducibly subjective – as, for example, with many of the normative and ethical issues associated with the practice of international relations. From various epistemological and analytical standpoints, single case study analysis can incorporate both idiographic sui generis cases and, where the potential for generalisation may exist, nomothetic case studies suitable for the testing and building of causal hypotheses. Finally, it should not be ignored that a signal advantage of the case study – with particular relevance to international relations – also exists at a more practical rather than theoretical level. This is, as Eckstein noted, “that it is economical for all resources: money, manpower, time, effort… especially important, of course, if studies are inherently costly, as they are if units are complex collective individuals ” (1975: 149-150, emphasis added).

Limitations

Single case study analysis has, however, been subject to a number of criticisms, the most common of which concern the inter-related issues of methodological rigour, researcher subjectivity, and external validity. With regard to the first point, the prototypical view here is that of Zeev Maoz (2002: 164-165), who suggests that “the use of the case study absolves the author from any kind of methodological considerations. Case studies have become in many cases a synonym for freeform research where anything goes”. The absence of systematic procedures for case study research is something that Yin (2009: 14-15) sees as traditionally the greatest concern due to a relative absence of methodological guidelines. As the previous section suggests, this critique seems somewhat unfair; many contemporary case study practitioners – and representing various strands of IR theory – have increasingly sought to clarify and develop their methodological techniques and epistemological grounding (Bennett and Elman, 2010: 499-500).

A second issue, again also incorporating issues of construct validity, concerns that of the reliability and replicability of various forms of single case study analysis. This is usually tied to a broader critique of qualitative research methods as a whole. However, whereas the latter obviously tend toward an explicitly-acknowledged interpretive basis for meanings, reasons, and understandings:

“quantitative measures appear objective, but only so long as we don’t ask questions about where and how the data were produced… pure objectivity is not a meaningful concept if the goal is to measure intangibles [as] these concepts only exist because we can interpret them” (Berg and Lune, 2010: 340).

The question of researcher subjectivity is a valid one, and it may be intended only as a methodological critique of what are obviously less formalised and researcher-independent methods (Verschuren, 2003). Owen (1994) and Layne’s (1994) contradictory process tracing results of interdemocratic war-avoidance during the Anglo-American crisis of 1861 to 1863 – from liberal and realist standpoints respectively – are a useful example. However, it does also rest on certain assumptions that can raise deeper and potentially irreconcilable ontological and epistemological issues. There are, regardless, plenty such as Bent Flyvbjerg (2006: 237) who suggest that the case study contains no greater bias toward verification than other methods of inquiry, and that “on the contrary, experience indicates that the case study contains a greater bias toward falsification of preconceived notions than toward verification”.

The third and arguably most prominent critique of single case study analysis is the issue of external validity or generalisability. How is it that one case can reliably offer anything beyond the particular? “We always do better (or, in the extreme, no worse) with more observation as the basis of our generalization”, as King et al write; “in all social science research and all prediction, it is important that we be as explicit as possible about the degree of uncertainty that accompanies out prediction” (1994: 212). This is an unavoidably valid criticism. It may be that theories which pass a single crucial case study test, for example, require rare antecedent conditions and therefore actually have little explanatory range. These conditions may emerge more clearly, as Van Evera (1997: 51-54) notes, from large-N studies in which cases that lack them present themselves as outliers exhibiting a theory’s cause but without its predicted outcome. As with the case of Indian democratisation above, it would logically be preferable to conduct large-N analysis beforehand to identify that state’s non-representative nature in relation to the broader population.

There are, however, three important qualifiers to the argument about generalisation that deserve particular mention here. The first is that with regard to an idiographic single-outcome case study, as Eckstein notes, the criticism is “mitigated by the fact that its capability to do so [is] never claimed by its exponents; in fact it is often explicitly repudiated” (1975: 134). Criticism of generalisability is of little relevance when the intention is one of particularisation. A second qualifier relates to the difference between statistical and analytical generalisation; single case studies are clearly less appropriate for the former but arguably retain significant utility for the latter – the difference also between explanatory and exploratory, or theory-testing and theory-building, as discussed above. As Gerring puts it, “theory confirmation/disconfirmation is not the case study’s strong suit” (2004: 350). A third qualification relates to the issue of case selection. As Seawright and Gerring (2008) note, the generalisability of case studies can be increased by the strategic selection of cases. Representative or random samples may not be the most appropriate, given that they may not provide the richest insight (or indeed, that a random and unknown deviant case may appear). Instead, and properly used , atypical or extreme cases “often reveal more information because they activate more actors… and more basic mechanisms in the situation studied” (Flyvbjerg, 2006). Of course, this also points to the very serious limitation, as hinted at with the case of India above, that poor case selection may alternatively lead to overgeneralisation and/or grievous misunderstandings of the relationship between variables or processes (Bennett and Elman, 2006a: 460-463).

As Tim May (2011: 226) notes, “the goal for many proponents of case studies […] is to overcome dichotomies between generalizing and particularizing, quantitative and qualitative, deductive and inductive techniques”. Research aims should drive methodological choices, rather than narrow and dogmatic preconceived approaches. As demonstrated above, there are various advantages to both idiographic and nomothetic single case study analyses – notably the empirically-rich, context-specific, holistic accounts that they have to offer, and their contribution to theory-building and, to a lesser extent, that of theory-testing. Furthermore, while they do possess clear limitations, any research method involves necessary trade-offs; the inherent weaknesses of any one method, however, can potentially be offset by situating them within a broader, pluralistic mixed-method research strategy. Whether or not single case studies are used in this fashion, they clearly have a great deal to offer.

References 

Bennett, A. and Checkel, J. T. (2012) ‘Process Tracing: From Philosophical Roots to Best Practice’, Simons Papers in Security and Development, No. 21/2012, School for International Studies, Simon Fraser University: Vancouver.

Bennett, A. and Elman, C. (2006a) ‘Qualitative Research: Recent Developments in Case Study Methods’, Annual Review of Political Science , 9, 455-476.

Bennett, A. and Elman, C. (2006b) ‘Complex Causal Relations and Case Study Methods: The Example of Path Dependence’, Political Analysis , 14, 3, 250-267.

Bennett, A. and Elman, C. (2007) ‘Case Study Methods in the International Relations Subfield’, Comparative Political Studies , 40, 2, 170-195.

Bennett, A. and Elman, C. (2010) Case Study Methods. In C. Reus-Smit and D. Snidal (eds) The Oxford Handbook of International Relations . Oxford University Press: Oxford. Ch. 29.

Berg, B. and Lune, H. (2012) Qualitative Research Methods for the Social Sciences . Pearson: London.

Bryman, A. (2012) Social Research Methods . Oxford University Press: Oxford.

David, M. and Sutton, C. D. (2011) Social Research: An Introduction . SAGE Publications Ltd: London.

Diamond, J. (1992) ‘Economic development and democracy reconsidered’, American Behavioral Scientist , 35, 4/5, 450-499.

Eckstein, H. (1975) Case Study and Theory in Political Science. In R. Gomm, M. Hammersley, and P. Foster (eds) Case Study Method . SAGE Publications Ltd: London.

Flyvbjerg, B. (2006) ‘Five Misunderstandings About Case-Study Research’, Qualitative Inquiry , 12, 2, 219-245.

Geertz, C. (1973) The Interpretation of Cultures: Selected Essays by Clifford Geertz . Basic Books Inc: New York.

Gerring, J. (2004) ‘What is a Case Study and What Is It Good for?’, American Political Science Review , 98, 2, 341-354.

Gerring, J. (2006a) Case Study Research: Principles and Practices . Cambridge University Press: Cambridge.

Gerring, J. (2006b) ‘Single-Outcome Studies: A Methodological Primer’, International Sociology , 21, 5, 707-734.

Gerring, J. (2007) ‘Is There a (Viable) Crucial-Case Method?’, Comparative Political Studies , 40, 3, 231-253.

King, G., Keohane, R. O. and Verba, S. (1994) Designing Social Inquiry: Scientific Inference in Qualitative Research . Princeton University Press: Chichester.

Layne, C. (1994) ‘Kant or Cant: The Myth of the Democratic Peace’, International Security , 19, 2, 5-49.

Levy, J. S. (2008) ‘Case Studies: Types, Designs, and Logics of Inference’, Conflict Management and Peace Science , 25, 1-18.

Lipset, S. M. (1959) ‘Some Social Requisites of Democracy: Economic Development and Political Legitimacy’, The American Political Science Review , 53, 1, 69-105.

Lyotard, J-F. (1984) The Postmodern Condition: A Report on Knowledge . University of Minnesota Press: Minneapolis.

MacMillan, A. (2008) ‘Deviant Democratization in India’, Democratization , 15, 4, 733-749.

Maoz, Z. (2002) Case study methodology in international studies: from storytelling to hypothesis testing. In F. P. Harvey and M. Brecher (eds) Evaluating Methodology in International Studies . University of Michigan Press: Ann Arbor.

May, T. (2011) Social Research: Issues, Methods and Process . Open University Press: Maidenhead.

Owen, J. M. (1994) ‘How Liberalism Produces Democratic Peace’, International Security , 19, 2, 87-125.

Seawright, J. and Gerring, J. (2008) ‘Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options’, Political Research Quarterly , 61, 2, 294-308.

Stake, R. E. (2008) Qualitative Case Studies. In N. K. Denzin and Y. S. Lincoln (eds) Strategies of Qualitative Inquiry . Sage Publications: Los Angeles. Ch. 17.

Van Evera, S. (1997) Guide to Methods for Students of Political Science . Cornell University Press: Ithaca.

Verschuren, P. J. M. (2003) ‘Case study as a research strategy: some ambiguities and opportunities’, International Journal of Social Research Methodology , 6, 2, 121-139.

Yin, R. K. (2009) Case Study Research: Design and Methods . SAGE Publications Ltd: London.

[1] The paper follows convention by differentiating between ‘International Relations’ as the academic discipline and ‘international relations’ as the subject of study.

[2] There is some similarity here with Stake’s (2008: 445-447) notion of intrinsic cases, those undertaken for a better understanding of the particular case, and instrumental ones that provide insight for the purposes of a wider external interest.

[3] These may be unique in the idiographic sense, or in nomothetic terms as an exception to the generalising suppositions of either probabilistic or deterministic theories (as per deviant cases, below).

[4] Although there are “philosophical hurdles to mount”, according to Bennett and Checkel, there exists no a priori reason as to why process tracing (as typically grounded in scientific realism) is fundamentally incompatible with various strands of positivism or interpretivism (2012: 18-19). By extension, it can therefore be incorporated by a range of contemporary mainstream IR theories.

— Written by: Ben Willis Written at: University of Plymouth Written for: David Brockington Date written: January 2013

Further Reading on E-International Relations

  • Identity in International Conflicts: A Case Study of the Cuban Missile Crisis
  • Imperialism’s Legacy in the Study of Contemporary Politics: The Case of Hegemonic Stability Theory
  • Recreating a Nation’s Identity Through Symbolism: A Chinese Case Study
  • Ontological Insecurity: A Case Study on Israeli-Palestinian Conflict in Jerusalem
  • Terrorists or Freedom Fighters: A Case Study of ETA
  • A Critical Assessment of Eco-Marxism: A Ghanaian Case Study

Please Consider Donating

Before you download your free e-book, please consider donating to support open access publishing.

E-IR is an independent non-profit publisher run by an all volunteer team. Your donations allow us to invest in new open access titles and pay our bandwidth bills to ensure we keep our existing titles free to view. Any amount, in any currency, is appreciated. Many thanks!

Donations are voluntary and not required to download the e-book - your link to download is below.

limitations of a qualitative case study

Qualitative study design: Case Studies

  • Qualitative study design
  • Phenomenology
  • Grounded theory
  • Ethnography
  • Narrative inquiry
  • Action research

Case Studies

  • Field research
  • Focus groups
  • Observation
  • Surveys & questionnaires
  • Study Designs Home

In depth description of the experience of a single person, a family, a group, a community or an organisation.

An example of a qualitative case study is a life history which is the story of one specific person.  A case study may be done to highlight a specific issue by telling a story of one person or one group. 

  • Oral recording

Ability to explore and describe, in depth, an issue or event. 

Develop an understanding of health, illness and health care in context. 

Single case can be used to develop or disprove a theory. 

Can be used as a model or prototype .  

Limitations

Labour intensive and generates large diverse data sets which can be hard to manage. 

Case studies are seen by many as a weak methodology because they only look at one person or one specific group and aren’t as broad in their participant selection as other methodologies. 

Example questions

This methodology can be used to ask questions about a specific drug or treatment and its effects on an individual.

  • Does thalidomide cause birth defects?
  • Does exposure to a pesticide lead to cancer?

Example studies

  • Choi, T. S. T., Walker, K. Z., & Palermo, C. (2018). Diabetes management in a foreign land: A case study on Chinese Australians. Health & Social Care in the Community, 26(2), e225-e232. 
  • Reade, I., Rodgers, W., & Spriggs, K. (2008). New Ideas for High Performance Coaches: A Case Study of Knowledge Transfer in Sport Science.  International Journal of Sports Science & Coaching , 3(3), 335-354. 
  • Wingrove, K., Barbour, L., & Palermo, C. (2017). Exploring nutrition capacity in Australia's charitable food sector.  Nutrition & Dietetics , 74(5), 495-501. 
  • Green, J., & Thorogood, N. (2018). Qualitative methods for health research (4th ed.). London: SAGE. 
  • University of Missouri-St. Louis. Qualitative Research Designs. Retrieved from http://www.umsl.edu/~lindquists/qualdsgn.html   
  • << Previous: Action research
  • Next: Field research >>
  • Last Updated: Apr 8, 2024 11:12 AM
  • URL: https://deakin.libguides.com/qualitative-study-designs

Case Study Research Method in Psychology

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews).

The case study research method originated in clinical medicine (the case history, i.e., the patient’s personal history). In psychology, case studies are often confined to the study of a particular individual.

The information is mainly biographical and relates to events in the individual’s past (i.e., retrospective), as well as to significant events that are currently occurring in his or her everyday life.

The case study is not a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies.

Freud (1909a, 1909b) conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

This makes it clear that the case study is a method that should only be used by a psychologist, therapist, or psychiatrist, i.e., someone with a professional qualification.

There is an ethical issue of competence. Only someone qualified to diagnose and treat a person can conduct a formal case study relating to atypical (i.e., abnormal) behavior or atypical development.

case study

 Famous Case Studies

  • Anna O – One of the most famous case studies, documenting psychoanalyst Josef Breuer’s treatment of “Anna O” (real name Bertha Pappenheim) for hysteria in the late 1800s using early psychoanalytic theory.
  • Little Hans – A child psychoanalysis case study published by Sigmund Freud in 1909 analyzing his five-year-old patient Herbert Graf’s house phobia as related to the Oedipus complex.
  • Bruce/Brenda – Gender identity case of the boy (Bruce) whose botched circumcision led psychologist John Money to advise gender reassignment and raise him as a girl (Brenda) in the 1960s.
  • Genie Wiley – Linguistics/psychological development case of the victim of extreme isolation abuse who was studied in 1970s California for effects of early language deprivation on acquiring speech later in life.
  • Phineas Gage – One of the most famous neuropsychology case studies analyzes personality changes in railroad worker Phineas Gage after an 1848 brain injury involving a tamping iron piercing his skull.

Clinical Case Studies

  • Studying the effectiveness of psychotherapy approaches with an individual patient
  • Assessing and treating mental illnesses like depression, anxiety disorders, PTSD
  • Neuropsychological cases investigating brain injuries or disorders

Child Psychology Case Studies

  • Studying psychological development from birth through adolescence
  • Cases of learning disabilities, autism spectrum disorders, ADHD
  • Effects of trauma, abuse, deprivation on development

Types of Case Studies

  • Explanatory case studies : Used to explore causation in order to find underlying principles. Helpful for doing qualitative analysis to explain presumed causal links.
  • Exploratory case studies : Used to explore situations where an intervention being evaluated has no clear set of outcomes. It helps define questions and hypotheses for future research.
  • Descriptive case studies : Describe an intervention or phenomenon and the real-life context in which it occurred. It is helpful for illustrating certain topics within an evaluation.
  • Multiple-case studies : Used to explore differences between cases and replicate findings across cases. Helpful for comparing and contrasting specific cases.
  • Intrinsic : Used to gain a better understanding of a particular case. Helpful for capturing the complexity of a single case.
  • Collective : Used to explore a general phenomenon using multiple case studies. Helpful for jointly studying a group of cases in order to inquire into the phenomenon.

Where Do You Find Data for a Case Study?

There are several places to find data for a case study. The key is to gather data from multiple sources to get a complete picture of the case and corroborate facts or findings through triangulation of evidence. Most of this information is likely qualitative (i.e., verbal description rather than measurement), but the psychologist might also collect numerical data.

1. Primary sources

  • Interviews – Interviewing key people related to the case to get their perspectives and insights. The interview is an extremely effective procedure for obtaining information about an individual, and it may be used to collect comments from the person’s friends, parents, employer, workmates, and others who have a good knowledge of the person, as well as to obtain facts from the person him or herself.
  • Observations – Observing behaviors, interactions, processes, etc., related to the case as they unfold in real-time.
  • Documents & Records – Reviewing private documents, diaries, public records, correspondence, meeting minutes, etc., relevant to the case.

2. Secondary sources

  • News/Media – News coverage of events related to the case study.
  • Academic articles – Journal articles, dissertations etc. that discuss the case.
  • Government reports – Official data and records related to the case context.
  • Books/films – Books, documentaries or films discussing the case.

3. Archival records

Searching historical archives, museum collections and databases to find relevant documents, visual/audio records related to the case history and context.

Public archives like newspapers, organizational records, photographic collections could all include potentially relevant pieces of information to shed light on attitudes, cultural perspectives, common practices and historical contexts related to psychology.

4. Organizational records

Organizational records offer the advantage of often having large datasets collected over time that can reveal or confirm psychological insights.

Of course, privacy and ethical concerns regarding confidential data must be navigated carefully.

However, with proper protocols, organizational records can provide invaluable context and empirical depth to qualitative case studies exploring the intersection of psychology and organizations.

  • Organizational/industrial psychology research : Organizational records like employee surveys, turnover/retention data, policies, incident reports etc. may provide insight into topics like job satisfaction, workplace culture and dynamics, leadership issues, employee behaviors etc.
  • Clinical psychology : Therapists/hospitals may grant access to anonymized medical records to study aspects like assessments, diagnoses, treatment plans etc. This could shed light on clinical practices.
  • School psychology : Studies could utilize anonymized student records like test scores, grades, disciplinary issues, and counseling referrals to study child development, learning barriers, effectiveness of support programs, and more.

How do I Write a Case Study in Psychology?

Follow specified case study guidelines provided by a journal or your psychology tutor. General components of clinical case studies include: background, symptoms, assessments, diagnosis, treatment, and outcomes. Interpreting the information means the researcher decides what to include or leave out. A good case study should always clarify which information is the factual description and which is an inference or the researcher’s opinion.

1. Introduction

  • Provide background on the case context and why it is of interest, presenting background information like demographics, relevant history, and presenting problem.
  • Compare briefly to similar published cases if applicable. Clearly state the focus/importance of the case.

2. Case Presentation

  • Describe the presenting problem in detail, including symptoms, duration,and impact on daily life.
  • Include client demographics like age and gender, information about social relationships, and mental health history.
  • Describe all physical, emotional, and/or sensory symptoms reported by the client.
  • Use patient quotes to describe the initial complaint verbatim. Follow with full-sentence summaries of relevant history details gathered, including key components that led to a working diagnosis.
  • Summarize clinical exam results, namely orthopedic/neurological tests, imaging, lab tests, etc. Note actual results rather than subjective conclusions. Provide images if clearly reproducible/anonymized.
  • Clearly state the working diagnosis or clinical impression before transitioning to management.

3. Management and Outcome

  • Indicate the total duration of care and number of treatments given over what timeframe. Use specific names/descriptions for any therapies/interventions applied.
  • Present the results of the intervention,including any quantitative or qualitative data collected.
  • For outcomes, utilize visual analog scales for pain, medication usage logs, etc., if possible. Include patient self-reports of improvement/worsening of symptoms. Note the reason for discharge/end of care.

4. Discussion

  • Analyze the case, exploring contributing factors, limitations of the study, and connections to existing research.
  • Analyze the effectiveness of the intervention,considering factors like participant adherence, limitations of the study, and potential alternative explanations for the results.
  • Identify any questions raised in the case analysis and relate insights to established theories and current research if applicable. Avoid definitive claims about physiological explanations.
  • Offer clinical implications, and suggest future research directions.

5. Additional Items

  • Thank specific assistants for writing support only. No patient acknowledgments.
  • References should directly support any key claims or quotes included.
  • Use tables/figures/images only if substantially informative. Include permissions and legends/explanatory notes.
  • Provides detailed (rich qualitative) information.
  • Provides insight for further research.
  • Permitting investigation of otherwise impractical (or unethical) situations.

Case studies allow a researcher to investigate a topic in far more detail than might be possible if they were trying to deal with a large number of research participants (nomothetic approach) with the aim of ‘averaging’.

Because of their in-depth, multi-sided approach, case studies often shed light on aspects of human thinking and behavior that would be unethical or impractical to study in other ways.

Research that only looks into the measurable aspects of human behavior is not likely to give us insights into the subjective dimension of experience, which is important to psychoanalytic and humanistic psychologists.

Case studies are often used in exploratory research. They can help us generate new ideas (that might be tested by other methods). They are an important way of illustrating theories and can help show how different aspects of a person’s life are related to each other.

The method is, therefore, important for psychologists who adopt a holistic point of view (i.e., humanistic psychologists ).

Limitations

  • Lacking scientific rigor and providing little basis for generalization of results to the wider population.
  • Researchers’ own subjective feelings may influence the case study (researcher bias).
  • Difficult to replicate.
  • Time-consuming and expensive.
  • The volume of data, together with the time restrictions in place, impacted the depth of analysis that was possible within the available resources.

Because a case study deals with only one person/event/group, we can never be sure if the case study investigated is representative of the wider body of “similar” instances. This means the conclusions drawn from a particular case may not be transferable to other settings.

Because case studies are based on the analysis of qualitative (i.e., descriptive) data , a lot depends on the psychologist’s interpretation of the information she has acquired.

This means that there is a lot of scope for Anna O , and it could be that the subjective opinions of the psychologist intrude in the assessment of what the data means.

For example, Freud has been criticized for producing case studies in which the information was sometimes distorted to fit particular behavioral theories (e.g., Little Hans ).

This is also true of Money’s interpretation of the Bruce/Brenda case study (Diamond, 1997) when he ignored evidence that went against his theory.

Breuer, J., & Freud, S. (1895).  Studies on hysteria . Standard Edition 2: London.

Curtiss, S. (1981). Genie: The case of a modern wild child .

Diamond, M., & Sigmundson, K. (1997). Sex Reassignment at Birth: Long-term Review and Clinical Implications. Archives of Pediatrics & Adolescent Medicine , 151(3), 298-304

Freud, S. (1909a). Analysis of a phobia of a five year old boy. In The Pelican Freud Library (1977), Vol 8, Case Histories 1, pages 169-306

Freud, S. (1909b). Bemerkungen über einen Fall von Zwangsneurose (Der “Rattenmann”). Jb. psychoanal. psychopathol. Forsch ., I, p. 357-421; GW, VII, p. 379-463; Notes upon a case of obsessional neurosis, SE , 10: 151-318.

Harlow J. M. (1848). Passage of an iron rod through the head.  Boston Medical and Surgical Journal, 39 , 389–393.

Harlow, J. M. (1868).  Recovery from the Passage of an Iron Bar through the Head .  Publications of the Massachusetts Medical Society. 2  (3), 327-347.

Money, J., & Ehrhardt, A. A. (1972).  Man & Woman, Boy & Girl : The Differentiation and Dimorphism of Gender Identity from Conception to Maturity. Baltimore, Maryland: Johns Hopkins University Press.

Money, J., & Tucker, P. (1975). Sexual signatures: On being a man or a woman.

Further Information

  • Case Study Approach
  • Case Study Method
  • Enhancing the Quality of Case Studies in Health Services Research
  • “We do things together” A case study of “couplehood” in dementia
  • Using mixed methods for evaluating an integrative approach to cancer care: a case study

Print Friendly, PDF & Email

  • First Online: 27 October 2022

Cite this chapter

Book cover

  • R. M. Channaveer 4 &
  • Rajendra Baikady 5  

2217 Accesses

1 Citations

This chapter reviews the strengths and limitations of case study as a research method in social sciences. It provides an account of an evidence base to justify why a case study is best suitable for some research questions and why not for some other research questions. Case study designing around the research context, defining the structure and modality, conducting the study, collecting the data through triangulation mode, analysing the data, and interpreting the data and theory building at the end give a holistic view of it. In addition, the chapter also focuses on the types of case study and when and where to use case study as a research method in social science research.

  • Qualitative research approach
  • Social work research

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
  • Available as EPUB and PDF
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
  • Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Ang, C. S., Lee, K. F., & Dipolog-Ubanan, G. F. (2019). Determinants of first-year student identity and satisfaction in higher education: A quantitative case study. SAGE Open, 9 (2), 215824401984668. https://doi.org/10.1177/2158244019846689

Baxter, P., & Jack, S. (2015). Qualitative case study methodology: Study design and implementation for novice researchers. The Qualitative Report . Published. https://doi.org/10.46743/2160-3715/2008.1573

Bhatta, T. P. (2018). Case study research, philosophical position and theory building: A methodological discussion. Dhaulagiri Journal of Sociology and Anthropology, 12 , 72–79. https://doi.org/10.3126/dsaj.v12i0.22182

Article   Google Scholar  

Bromley, P. D. (1990). Academic contributions to psychological counselling. A philosophy of science for the study of individual cases. Counselling Psychology Quarterly , 3 (3), 299–307.

Google Scholar  

Crowe, S., Cresswell, K., Robertson, A., Huby, G., Avery, A., & Sheikh, A. (2011). The case study approach. BMC Medical Research Methodology, 11 (1), 1–9.

Grässel, E., & Schirmer, B. (2006). The use of volunteers to support family carers of dementia patients: Results of a prospective longitudinal study investigating expectations towards and experience with training and professional support. Zeitschrift Fur Gerontologie Und Geriatrie, 39 (3), 217–226.

Greenwood, D., & Lowenthal, D. (2005). Case study as a means of researching social work and improving practitioner education. Journal of Social Work Practice, 19 (2), 181–193. https://doi.org/10.1080/02650530500144782

Gülseçen, S., & Kubat, A. (2006). Teaching ICT to teacher candidates using PBL: A qualitative and quantitative evaluation. Journal of Educational Technology & Society, 9 (2), 96–106.

Gomm, R., Hammersley, M., & Foster, P. (2000). Case study and generalization. Case study method , 98–115.

Hamera, J., Denzin, N. K., & Lincoln, Y. S. (2011). Performance ethnography . SAGE.

Hayes, N. (2000). Doing psychological research (p. 133). Open University Press.

Harrison, H., Birks, M., Franklin, R., & Mills, J. (2017). Case study research: Foundations and methodological orientations. In Forum qualitative sozialforschung/forum: Qualitative social research (Vol. 18, No. 1).

Iwakabe, S., & Gazzola, N. (2009). From single-case studies to practice-based knowledge: Aggregating and synthesizing case studies. Psychotherapy Research, 19 (4–5), 601–611. https://doi.org/10.1080/10503300802688494

Johnson, M. P. (2006). Decision models for the location of community corrections centers. Environment and Planning b: Planning and Design, 33 (3), 393–412. https://doi.org/10.1068/b3125

Kaarbo, J., & Beasley, R. K. (1999). A practical guide to the comparative case study method in political psychology. Political Psychology, 20 (2), 369–391. https://doi.org/10.1111/0162-895x.00149

Lovell, G. I. (2006). Justice excused: The deployment of law in everyday political encounters. Law Society Review, 40 (2), 283–324. https://doi.org/10.1111/j.1540-5893.2006.00265.x

McDonough, S., & McDonough, S. (1997). Research methods as part of English language teacher education. English Language Teacher Education and Development, 3 (1), 84–96.

Meredith, J. (1998). Building operations management theory through case and field research. Journal of Operations Management, 16 (4), 441–454. https://doi.org/10.1016/s0272-6963(98)00023-0

Mills, A. J., Durepos, G., & Wiebe, E. (Eds.). (2009). Encyclopedia of case study research . Sage Publications.

Ochieng, P. A. (2009). An analysis of the strengths and limitation of qualitative and quantitative research paradigms. Problems of Education in the 21st Century , 13 , 13.

Page, E. B., Webb, E. J., Campell, D. T., Schwart, R. D., & Sechrest, L. (1966). Unobtrusive measures: Nonreactive research in the social sciences. American Educational Research Journal, 3 (4), 317. https://doi.org/10.2307/1162043

Rashid, Y., Rashid, A., Warraich, M. A., Sabir, S. S., & Waseem, A. (2019). Case study method: A step-by-step guide for business researchers. International Journal of Qualitative Methods, 18 , 160940691986242. https://doi.org/10.1177/1609406919862424

Ridder, H. G. (2017). The theory contribution of case study research designs. Business Research, 10 (2), 281–305. https://doi.org/10.1007/s40685-017-0045-z

Sadeghi Moghadam, M. R., Ghasemnia Arabi, N., & Khoshsima, G. (2021). A Review of case study method in operations management research. International Journal of Qualitative Methods, 20 , 160940692110100. https://doi.org/10.1177/16094069211010088

Sommer, B. B., & Sommer, R. (1997). A practical guide to behavioral research: Tools and techniques . Oxford University Press.

Stake, R. E. (2010). Qualitative research: Studying how things work .

Stake, R. E. (1995). The Art of Case Study Research . Sage Publications.

Stoecker, R. (1991). Evaluating and rethinking the case study. The Sociological Review, 39 (1), 88–112.

Suryani, A. (2013). Comparing case study and ethnography as qualitative research approaches .

Taylor, S., & Berridge, V. (2006). Medicinal plants and malaria: An historical case study of research at the London School of Hygiene and Tropical Medicine in the twentieth century. Transactions of the Royal Society of Tropical Medicine and Hygiene, 100 (8), 707–714. https://doi.org/10.1016/j.trstmh.2005.11.017

Tellis, W. (1997). Introduction to case study. The Qualitative Report . Published. https://doi.org/10.46743/2160-3715/1997.2024

Towne, L., & Shavelson, R. J. (2002). Scientific research in education . National Academy Press Publications Sales Office.

Widdowson, M. D. J. (2011). Case study research methodology. International Journal of Transactional Analysis Research, 2 (1), 25–34.

Yin, R. K. (2004). The case study anthology . Sage.

Yin, R. K. (2003). Design and methods. Case Study Research , 3 (9.2).

Yin, R. K. (1994). Case study research: Design and methods (2nd ed.). Sage Publishing.

Yin, R. (1984). Case study research: Design and methods . Sage Publications Beverly Hills.

Yin, R. (1993). Applications of case study research . Sage Publishing.

Zainal, Z. (2003). An investigation into the effects of discipline-specific knowledge, proficiency and genre on reading comprehension and strategies of Malaysia ESP Students. Unpublished Ph. D. Thesis. University of Reading , 1 (1).

Zeisel, J. (1984). Inquiry by design: Tools for environment-behaviour research (No. 5). CUP archive.

Download references

Author information

Authors and affiliations.

Department of Social Work, Central University of Karnataka, Kadaganchi, India

R. M. Channaveer

Department of Social Work, University of Johannesburg, Johannesburg, South Africa

Rajendra Baikady

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to R. M. Channaveer .

Editor information

Editors and affiliations.

Centre for Family and Child Studies, Research Institute of Humanities and Social Sciences, University of Sharjah, Sharjah, United Arab Emirates

M. Rezaul Islam

Department of Development Studies, University of Dhaka, Dhaka, Bangladesh

Niaz Ahmed Khan

Department of Social Work, School of Humanities, University of Johannesburg, Johannesburg, South Africa

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Channaveer, R.M., Baikady, R. (2022). Case Study. In: Islam, M.R., Khan, N.A., Baikady, R. (eds) Principles of Social Research Methodology. Springer, Singapore. https://doi.org/10.1007/978-981-19-5441-2_21

Download citation

DOI : https://doi.org/10.1007/978-981-19-5441-2_21

Published : 27 October 2022

Publisher Name : Springer, Singapore

Print ISBN : 978-981-19-5219-7

Online ISBN : 978-981-19-5441-2

eBook Packages : Social Sciences

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Publish with us

Policies and ethics

  • Find a journal
  • Track your research
  • Privacy Policy

Buy Me a Coffee

Research Method

Home » Case Study – Methods, Examples and Guide

Case Study – Methods, Examples and Guide

Table of Contents

Case Study Research

A case study is a research method that involves an in-depth examination and analysis of a particular phenomenon or case, such as an individual, organization, community, event, or situation.

It is a qualitative research approach that aims to provide a detailed and comprehensive understanding of the case being studied. Case studies typically involve multiple sources of data, including interviews, observations, documents, and artifacts, which are analyzed using various techniques, such as content analysis, thematic analysis, and grounded theory. The findings of a case study are often used to develop theories, inform policy or practice, or generate new research questions.

Types of Case Study

Types and Methods of Case Study are as follows:

Single-Case Study

A single-case study is an in-depth analysis of a single case. This type of case study is useful when the researcher wants to understand a specific phenomenon in detail.

For Example , A researcher might conduct a single-case study on a particular individual to understand their experiences with a particular health condition or a specific organization to explore their management practices. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a single-case study are often used to generate new research questions, develop theories, or inform policy or practice.

Multiple-Case Study

A multiple-case study involves the analysis of several cases that are similar in nature. This type of case study is useful when the researcher wants to identify similarities and differences between the cases.

For Example, a researcher might conduct a multiple-case study on several companies to explore the factors that contribute to their success or failure. The researcher collects data from each case, compares and contrasts the findings, and uses various techniques to analyze the data, such as comparative analysis or pattern-matching. The findings of a multiple-case study can be used to develop theories, inform policy or practice, or generate new research questions.

Exploratory Case Study

An exploratory case study is used to explore a new or understudied phenomenon. This type of case study is useful when the researcher wants to generate hypotheses or theories about the phenomenon.

For Example, a researcher might conduct an exploratory case study on a new technology to understand its potential impact on society. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as grounded theory or content analysis. The findings of an exploratory case study can be used to generate new research questions, develop theories, or inform policy or practice.

Descriptive Case Study

A descriptive case study is used to describe a particular phenomenon in detail. This type of case study is useful when the researcher wants to provide a comprehensive account of the phenomenon.

For Example, a researcher might conduct a descriptive case study on a particular community to understand its social and economic characteristics. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a descriptive case study can be used to inform policy or practice or generate new research questions.

Instrumental Case Study

An instrumental case study is used to understand a particular phenomenon that is instrumental in achieving a particular goal. This type of case study is useful when the researcher wants to understand the role of the phenomenon in achieving the goal.

For Example, a researcher might conduct an instrumental case study on a particular policy to understand its impact on achieving a particular goal, such as reducing poverty. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of an instrumental case study can be used to inform policy or practice or generate new research questions.

Case Study Data Collection Methods

Here are some common data collection methods for case studies:

Interviews involve asking questions to individuals who have knowledge or experience relevant to the case study. Interviews can be structured (where the same questions are asked to all participants) or unstructured (where the interviewer follows up on the responses with further questions). Interviews can be conducted in person, over the phone, or through video conferencing.

Observations

Observations involve watching and recording the behavior and activities of individuals or groups relevant to the case study. Observations can be participant (where the researcher actively participates in the activities) or non-participant (where the researcher observes from a distance). Observations can be recorded using notes, audio or video recordings, or photographs.

Documents can be used as a source of information for case studies. Documents can include reports, memos, emails, letters, and other written materials related to the case study. Documents can be collected from the case study participants or from public sources.

Surveys involve asking a set of questions to a sample of individuals relevant to the case study. Surveys can be administered in person, over the phone, through mail or email, or online. Surveys can be used to gather information on attitudes, opinions, or behaviors related to the case study.

Artifacts are physical objects relevant to the case study. Artifacts can include tools, equipment, products, or other objects that provide insights into the case study phenomenon.

How to conduct Case Study Research

Conducting a case study research involves several steps that need to be followed to ensure the quality and rigor of the study. Here are the steps to conduct case study research:

  • Define the research questions: The first step in conducting a case study research is to define the research questions. The research questions should be specific, measurable, and relevant to the case study phenomenon under investigation.
  • Select the case: The next step is to select the case or cases to be studied. The case should be relevant to the research questions and should provide rich and diverse data that can be used to answer the research questions.
  • Collect data: Data can be collected using various methods, such as interviews, observations, documents, surveys, and artifacts. The data collection method should be selected based on the research questions and the nature of the case study phenomenon.
  • Analyze the data: The data collected from the case study should be analyzed using various techniques, such as content analysis, thematic analysis, or grounded theory. The analysis should be guided by the research questions and should aim to provide insights and conclusions relevant to the research questions.
  • Draw conclusions: The conclusions drawn from the case study should be based on the data analysis and should be relevant to the research questions. The conclusions should be supported by evidence and should be clearly stated.
  • Validate the findings: The findings of the case study should be validated by reviewing the data and the analysis with participants or other experts in the field. This helps to ensure the validity and reliability of the findings.
  • Write the report: The final step is to write the report of the case study research. The report should provide a clear description of the case study phenomenon, the research questions, the data collection methods, the data analysis, the findings, and the conclusions. The report should be written in a clear and concise manner and should follow the guidelines for academic writing.

Examples of Case Study

Here are some examples of case study research:

  • The Hawthorne Studies : Conducted between 1924 and 1932, the Hawthorne Studies were a series of case studies conducted by Elton Mayo and his colleagues to examine the impact of work environment on employee productivity. The studies were conducted at the Hawthorne Works plant of the Western Electric Company in Chicago and included interviews, observations, and experiments.
  • The Stanford Prison Experiment: Conducted in 1971, the Stanford Prison Experiment was a case study conducted by Philip Zimbardo to examine the psychological effects of power and authority. The study involved simulating a prison environment and assigning participants to the role of guards or prisoners. The study was controversial due to the ethical issues it raised.
  • The Challenger Disaster: The Challenger Disaster was a case study conducted to examine the causes of the Space Shuttle Challenger explosion in 1986. The study included interviews, observations, and analysis of data to identify the technical, organizational, and cultural factors that contributed to the disaster.
  • The Enron Scandal: The Enron Scandal was a case study conducted to examine the causes of the Enron Corporation’s bankruptcy in 2001. The study included interviews, analysis of financial data, and review of documents to identify the accounting practices, corporate culture, and ethical issues that led to the company’s downfall.
  • The Fukushima Nuclear Disaster : The Fukushima Nuclear Disaster was a case study conducted to examine the causes of the nuclear accident that occurred at the Fukushima Daiichi Nuclear Power Plant in Japan in 2011. The study included interviews, analysis of data, and review of documents to identify the technical, organizational, and cultural factors that contributed to the disaster.

Application of Case Study

Case studies have a wide range of applications across various fields and industries. Here are some examples:

Business and Management

Case studies are widely used in business and management to examine real-life situations and develop problem-solving skills. Case studies can help students and professionals to develop a deep understanding of business concepts, theories, and best practices.

Case studies are used in healthcare to examine patient care, treatment options, and outcomes. Case studies can help healthcare professionals to develop critical thinking skills, diagnose complex medical conditions, and develop effective treatment plans.

Case studies are used in education to examine teaching and learning practices. Case studies can help educators to develop effective teaching strategies, evaluate student progress, and identify areas for improvement.

Social Sciences

Case studies are widely used in social sciences to examine human behavior, social phenomena, and cultural practices. Case studies can help researchers to develop theories, test hypotheses, and gain insights into complex social issues.

Law and Ethics

Case studies are used in law and ethics to examine legal and ethical dilemmas. Case studies can help lawyers, policymakers, and ethical professionals to develop critical thinking skills, analyze complex cases, and make informed decisions.

Purpose of Case Study

The purpose of a case study is to provide a detailed analysis of a specific phenomenon, issue, or problem in its real-life context. A case study is a qualitative research method that involves the in-depth exploration and analysis of a particular case, which can be an individual, group, organization, event, or community.

The primary purpose of a case study is to generate a comprehensive and nuanced understanding of the case, including its history, context, and dynamics. Case studies can help researchers to identify and examine the underlying factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and detailed understanding of the case, which can inform future research, practice, or policy.

Case studies can also serve other purposes, including:

  • Illustrating a theory or concept: Case studies can be used to illustrate and explain theoretical concepts and frameworks, providing concrete examples of how they can be applied in real-life situations.
  • Developing hypotheses: Case studies can help to generate hypotheses about the causal relationships between different factors and outcomes, which can be tested through further research.
  • Providing insight into complex issues: Case studies can provide insights into complex and multifaceted issues, which may be difficult to understand through other research methods.
  • Informing practice or policy: Case studies can be used to inform practice or policy by identifying best practices, lessons learned, or areas for improvement.

Advantages of Case Study Research

There are several advantages of case study research, including:

  • In-depth exploration: Case study research allows for a detailed exploration and analysis of a specific phenomenon, issue, or problem in its real-life context. This can provide a comprehensive understanding of the case and its dynamics, which may not be possible through other research methods.
  • Rich data: Case study research can generate rich and detailed data, including qualitative data such as interviews, observations, and documents. This can provide a nuanced understanding of the case and its complexity.
  • Holistic perspective: Case study research allows for a holistic perspective of the case, taking into account the various factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and comprehensive understanding of the case.
  • Theory development: Case study research can help to develop and refine theories and concepts by providing empirical evidence and concrete examples of how they can be applied in real-life situations.
  • Practical application: Case study research can inform practice or policy by identifying best practices, lessons learned, or areas for improvement.
  • Contextualization: Case study research takes into account the specific context in which the case is situated, which can help to understand how the case is influenced by the social, cultural, and historical factors of its environment.

Limitations of Case Study Research

There are several limitations of case study research, including:

  • Limited generalizability : Case studies are typically focused on a single case or a small number of cases, which limits the generalizability of the findings. The unique characteristics of the case may not be applicable to other contexts or populations, which may limit the external validity of the research.
  • Biased sampling: Case studies may rely on purposive or convenience sampling, which can introduce bias into the sample selection process. This may limit the representativeness of the sample and the generalizability of the findings.
  • Subjectivity: Case studies rely on the interpretation of the researcher, which can introduce subjectivity into the analysis. The researcher’s own biases, assumptions, and perspectives may influence the findings, which may limit the objectivity of the research.
  • Limited control: Case studies are typically conducted in naturalistic settings, which limits the control that the researcher has over the environment and the variables being studied. This may limit the ability to establish causal relationships between variables.
  • Time-consuming: Case studies can be time-consuming to conduct, as they typically involve a detailed exploration and analysis of a specific case. This may limit the feasibility of conducting multiple case studies or conducting case studies in a timely manner.
  • Resource-intensive: Case studies may require significant resources, including time, funding, and expertise. This may limit the ability of researchers to conduct case studies in resource-constrained settings.

About the author

' src=

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Questionnaire

Questionnaire – Definition, Types, and Examples

Observational Research

Observational Research – Methods and Guide

Quantitative Research

Quantitative Research – Methods, Types and...

Qualitative Research Methods

Qualitative Research Methods

Explanatory Research

Explanatory Research – Types, Methods, Guide

Survey Research

Survey Research – Types, Methods, Examples

  • Open access
  • Published: 27 June 2011

The case study approach

  • Sarah Crowe 1 ,
  • Kathrin Cresswell 2 ,
  • Ann Robertson 2 ,
  • Guro Huby 3 ,
  • Anthony Avery 1 &
  • Aziz Sheikh 2  

BMC Medical Research Methodology volume  11 , Article number:  100 ( 2011 ) Cite this article

765k Accesses

1031 Citations

37 Altmetric

Metrics details

The case study approach allows in-depth, multi-faceted explorations of complex issues in their real-life settings. The value of the case study approach is well recognised in the fields of business, law and policy, but somewhat less so in health services research. Based on our experiences of conducting several health-related case studies, we reflect on the different types of case study design, the specific research questions this approach can help answer, the data sources that tend to be used, and the particular advantages and disadvantages of employing this methodological approach. The paper concludes with key pointers to aid those designing and appraising proposals for conducting case study research, and a checklist to help readers assess the quality of case study reports.

Peer Review reports

Introduction

The case study approach is particularly useful to employ when there is a need to obtain an in-depth appreciation of an issue, event or phenomenon of interest, in its natural real-life context. Our aim in writing this piece is to provide insights into when to consider employing this approach and an overview of key methodological considerations in relation to the design, planning, analysis, interpretation and reporting of case studies.

The illustrative 'grand round', 'case report' and 'case series' have a long tradition in clinical practice and research. Presenting detailed critiques, typically of one or more patients, aims to provide insights into aspects of the clinical case and, in doing so, illustrate broader lessons that may be learnt. In research, the conceptually-related case study approach can be used, for example, to describe in detail a patient's episode of care, explore professional attitudes to and experiences of a new policy initiative or service development or more generally to 'investigate contemporary phenomena within its real-life context' [ 1 ]. Based on our experiences of conducting a range of case studies, we reflect on when to consider using this approach, discuss the key steps involved and illustrate, with examples, some of the practical challenges of attaining an in-depth understanding of a 'case' as an integrated whole. In keeping with previously published work, we acknowledge the importance of theory to underpin the design, selection, conduct and interpretation of case studies[ 2 ]. In so doing, we make passing reference to the different epistemological approaches used in case study research by key theoreticians and methodologists in this field of enquiry.

This paper is structured around the following main questions: What is a case study? What are case studies used for? How are case studies conducted? What are the potential pitfalls and how can these be avoided? We draw in particular on four of our own recently published examples of case studies (see Tables 1 , 2 , 3 and 4 ) and those of others to illustrate our discussion[ 3 – 7 ].

What is a case study?

A case study is a research approach that is used to generate an in-depth, multi-faceted understanding of a complex issue in its real-life context. It is an established research design that is used extensively in a wide variety of disciplines, particularly in the social sciences. A case study can be defined in a variety of ways (Table 5 ), the central tenet being the need to explore an event or phenomenon in depth and in its natural context. It is for this reason sometimes referred to as a "naturalistic" design; this is in contrast to an "experimental" design (such as a randomised controlled trial) in which the investigator seeks to exert control over and manipulate the variable(s) of interest.

Stake's work has been particularly influential in defining the case study approach to scientific enquiry. He has helpfully characterised three main types of case study: intrinsic , instrumental and collective [ 8 ]. An intrinsic case study is typically undertaken to learn about a unique phenomenon. The researcher should define the uniqueness of the phenomenon, which distinguishes it from all others. In contrast, the instrumental case study uses a particular case (some of which may be better than others) to gain a broader appreciation of an issue or phenomenon. The collective case study involves studying multiple cases simultaneously or sequentially in an attempt to generate a still broader appreciation of a particular issue.

These are however not necessarily mutually exclusive categories. In the first of our examples (Table 1 ), we undertook an intrinsic case study to investigate the issue of recruitment of minority ethnic people into the specific context of asthma research studies, but it developed into a instrumental case study through seeking to understand the issue of recruitment of these marginalised populations more generally, generating a number of the findings that are potentially transferable to other disease contexts[ 3 ]. In contrast, the other three examples (see Tables 2 , 3 and 4 ) employed collective case study designs to study the introduction of workforce reconfiguration in primary care, the implementation of electronic health records into hospitals, and to understand the ways in which healthcare students learn about patient safety considerations[ 4 – 6 ]. Although our study focusing on the introduction of General Practitioners with Specialist Interests (Table 2 ) was explicitly collective in design (four contrasting primary care organisations were studied), is was also instrumental in that this particular professional group was studied as an exemplar of the more general phenomenon of workforce redesign[ 4 ].

What are case studies used for?

According to Yin, case studies can be used to explain, describe or explore events or phenomena in the everyday contexts in which they occur[ 1 ]. These can, for example, help to understand and explain causal links and pathways resulting from a new policy initiative or service development (see Tables 2 and 3 , for example)[ 1 ]. In contrast to experimental designs, which seek to test a specific hypothesis through deliberately manipulating the environment (like, for example, in a randomised controlled trial giving a new drug to randomly selected individuals and then comparing outcomes with controls),[ 9 ] the case study approach lends itself well to capturing information on more explanatory ' how ', 'what' and ' why ' questions, such as ' how is the intervention being implemented and received on the ground?'. The case study approach can offer additional insights into what gaps exist in its delivery or why one implementation strategy might be chosen over another. This in turn can help develop or refine theory, as shown in our study of the teaching of patient safety in undergraduate curricula (Table 4 )[ 6 , 10 ]. Key questions to consider when selecting the most appropriate study design are whether it is desirable or indeed possible to undertake a formal experimental investigation in which individuals and/or organisations are allocated to an intervention or control arm? Or whether the wish is to obtain a more naturalistic understanding of an issue? The former is ideally studied using a controlled experimental design, whereas the latter is more appropriately studied using a case study design.

Case studies may be approached in different ways depending on the epistemological standpoint of the researcher, that is, whether they take a critical (questioning one's own and others' assumptions), interpretivist (trying to understand individual and shared social meanings) or positivist approach (orientating towards the criteria of natural sciences, such as focusing on generalisability considerations) (Table 6 ). Whilst such a schema can be conceptually helpful, it may be appropriate to draw on more than one approach in any case study, particularly in the context of conducting health services research. Doolin has, for example, noted that in the context of undertaking interpretative case studies, researchers can usefully draw on a critical, reflective perspective which seeks to take into account the wider social and political environment that has shaped the case[ 11 ].

How are case studies conducted?

Here, we focus on the main stages of research activity when planning and undertaking a case study; the crucial stages are: defining the case; selecting the case(s); collecting and analysing the data; interpreting data; and reporting the findings.

Defining the case

Carefully formulated research question(s), informed by the existing literature and a prior appreciation of the theoretical issues and setting(s), are all important in appropriately and succinctly defining the case[ 8 , 12 ]. Crucially, each case should have a pre-defined boundary which clarifies the nature and time period covered by the case study (i.e. its scope, beginning and end), the relevant social group, organisation or geographical area of interest to the investigator, the types of evidence to be collected, and the priorities for data collection and analysis (see Table 7 )[ 1 ]. A theory driven approach to defining the case may help generate knowledge that is potentially transferable to a range of clinical contexts and behaviours; using theory is also likely to result in a more informed appreciation of, for example, how and why interventions have succeeded or failed[ 13 ].

For example, in our evaluation of the introduction of electronic health records in English hospitals (Table 3 ), we defined our cases as the NHS Trusts that were receiving the new technology[ 5 ]. Our focus was on how the technology was being implemented. However, if the primary research interest had been on the social and organisational dimensions of implementation, we might have defined our case differently as a grouping of healthcare professionals (e.g. doctors and/or nurses). The precise beginning and end of the case may however prove difficult to define. Pursuing this same example, when does the process of implementation and adoption of an electronic health record system really begin or end? Such judgements will inevitably be influenced by a range of factors, including the research question, theory of interest, the scope and richness of the gathered data and the resources available to the research team.

Selecting the case(s)

The decision on how to select the case(s) to study is a very important one that merits some reflection. In an intrinsic case study, the case is selected on its own merits[ 8 ]. The case is selected not because it is representative of other cases, but because of its uniqueness, which is of genuine interest to the researchers. This was, for example, the case in our study of the recruitment of minority ethnic participants into asthma research (Table 1 ) as our earlier work had demonstrated the marginalisation of minority ethnic people with asthma, despite evidence of disproportionate asthma morbidity[ 14 , 15 ]. In another example of an intrinsic case study, Hellstrom et al.[ 16 ] studied an elderly married couple living with dementia to explore how dementia had impacted on their understanding of home, their everyday life and their relationships.

For an instrumental case study, selecting a "typical" case can work well[ 8 ]. In contrast to the intrinsic case study, the particular case which is chosen is of less importance than selecting a case that allows the researcher to investigate an issue or phenomenon. For example, in order to gain an understanding of doctors' responses to health policy initiatives, Som undertook an instrumental case study interviewing clinicians who had a range of responsibilities for clinical governance in one NHS acute hospital trust[ 17 ]. Sampling a "deviant" or "atypical" case may however prove even more informative, potentially enabling the researcher to identify causal processes, generate hypotheses and develop theory.

In collective or multiple case studies, a number of cases are carefully selected. This offers the advantage of allowing comparisons to be made across several cases and/or replication. Choosing a "typical" case may enable the findings to be generalised to theory (i.e. analytical generalisation) or to test theory by replicating the findings in a second or even a third case (i.e. replication logic)[ 1 ]. Yin suggests two or three literal replications (i.e. predicting similar results) if the theory is straightforward and five or more if the theory is more subtle. However, critics might argue that selecting 'cases' in this way is insufficiently reflexive and ill-suited to the complexities of contemporary healthcare organisations.

The selected case study site(s) should allow the research team access to the group of individuals, the organisation, the processes or whatever else constitutes the chosen unit of analysis for the study. Access is therefore a central consideration; the researcher needs to come to know the case study site(s) well and to work cooperatively with them. Selected cases need to be not only interesting but also hospitable to the inquiry [ 8 ] if they are to be informative and answer the research question(s). Case study sites may also be pre-selected for the researcher, with decisions being influenced by key stakeholders. For example, our selection of case study sites in the evaluation of the implementation and adoption of electronic health record systems (see Table 3 ) was heavily influenced by NHS Connecting for Health, the government agency that was responsible for overseeing the National Programme for Information Technology (NPfIT)[ 5 ]. This prominent stakeholder had already selected the NHS sites (through a competitive bidding process) to be early adopters of the electronic health record systems and had negotiated contracts that detailed the deployment timelines.

It is also important to consider in advance the likely burden and risks associated with participation for those who (or the site(s) which) comprise the case study. Of particular importance is the obligation for the researcher to think through the ethical implications of the study (e.g. the risk of inadvertently breaching anonymity or confidentiality) and to ensure that potential participants/participating sites are provided with sufficient information to make an informed choice about joining the study. The outcome of providing this information might be that the emotive burden associated with participation, or the organisational disruption associated with supporting the fieldwork, is considered so high that the individuals or sites decide against participation.

In our example of evaluating implementations of electronic health record systems, given the restricted number of early adopter sites available to us, we sought purposively to select a diverse range of implementation cases among those that were available[ 5 ]. We chose a mixture of teaching, non-teaching and Foundation Trust hospitals, and examples of each of the three electronic health record systems procured centrally by the NPfIT. At one recruited site, it quickly became apparent that access was problematic because of competing demands on that organisation. Recognising the importance of full access and co-operative working for generating rich data, the research team decided not to pursue work at that site and instead to focus on other recruited sites.

Collecting the data

In order to develop a thorough understanding of the case, the case study approach usually involves the collection of multiple sources of evidence, using a range of quantitative (e.g. questionnaires, audits and analysis of routinely collected healthcare data) and more commonly qualitative techniques (e.g. interviews, focus groups and observations). The use of multiple sources of data (data triangulation) has been advocated as a way of increasing the internal validity of a study (i.e. the extent to which the method is appropriate to answer the research question)[ 8 , 18 – 21 ]. An underlying assumption is that data collected in different ways should lead to similar conclusions, and approaching the same issue from different angles can help develop a holistic picture of the phenomenon (Table 2 )[ 4 ].

Brazier and colleagues used a mixed-methods case study approach to investigate the impact of a cancer care programme[ 22 ]. Here, quantitative measures were collected with questionnaires before, and five months after, the start of the intervention which did not yield any statistically significant results. Qualitative interviews with patients however helped provide an insight into potentially beneficial process-related aspects of the programme, such as greater, perceived patient involvement in care. The authors reported how this case study approach provided a number of contextual factors likely to influence the effectiveness of the intervention and which were not likely to have been obtained from quantitative methods alone.

In collective or multiple case studies, data collection needs to be flexible enough to allow a detailed description of each individual case to be developed (e.g. the nature of different cancer care programmes), before considering the emerging similarities and differences in cross-case comparisons (e.g. to explore why one programme is more effective than another). It is important that data sources from different cases are, where possible, broadly comparable for this purpose even though they may vary in nature and depth.

Analysing, interpreting and reporting case studies

Making sense and offering a coherent interpretation of the typically disparate sources of data (whether qualitative alone or together with quantitative) is far from straightforward. Repeated reviewing and sorting of the voluminous and detail-rich data are integral to the process of analysis. In collective case studies, it is helpful to analyse data relating to the individual component cases first, before making comparisons across cases. Attention needs to be paid to variations within each case and, where relevant, the relationship between different causes, effects and outcomes[ 23 ]. Data will need to be organised and coded to allow the key issues, both derived from the literature and emerging from the dataset, to be easily retrieved at a later stage. An initial coding frame can help capture these issues and can be applied systematically to the whole dataset with the aid of a qualitative data analysis software package.

The Framework approach is a practical approach, comprising of five stages (familiarisation; identifying a thematic framework; indexing; charting; mapping and interpretation) , to managing and analysing large datasets particularly if time is limited, as was the case in our study of recruitment of South Asians into asthma research (Table 1 )[ 3 , 24 ]. Theoretical frameworks may also play an important role in integrating different sources of data and examining emerging themes. For example, we drew on a socio-technical framework to help explain the connections between different elements - technology; people; and the organisational settings within which they worked - in our study of the introduction of electronic health record systems (Table 3 )[ 5 ]. Our study of patient safety in undergraduate curricula drew on an evaluation-based approach to design and analysis, which emphasised the importance of the academic, organisational and practice contexts through which students learn (Table 4 )[ 6 ].

Case study findings can have implications both for theory development and theory testing. They may establish, strengthen or weaken historical explanations of a case and, in certain circumstances, allow theoretical (as opposed to statistical) generalisation beyond the particular cases studied[ 12 ]. These theoretical lenses should not, however, constitute a strait-jacket and the cases should not be "forced to fit" the particular theoretical framework that is being employed.

When reporting findings, it is important to provide the reader with enough contextual information to understand the processes that were followed and how the conclusions were reached. In a collective case study, researchers may choose to present the findings from individual cases separately before amalgamating across cases. Care must be taken to ensure the anonymity of both case sites and individual participants (if agreed in advance) by allocating appropriate codes or withholding descriptors. In the example given in Table 3 , we decided against providing detailed information on the NHS sites and individual participants in order to avoid the risk of inadvertent disclosure of identities[ 5 , 25 ].

What are the potential pitfalls and how can these be avoided?

The case study approach is, as with all research, not without its limitations. When investigating the formal and informal ways undergraduate students learn about patient safety (Table 4 ), for example, we rapidly accumulated a large quantity of data. The volume of data, together with the time restrictions in place, impacted on the depth of analysis that was possible within the available resources. This highlights a more general point of the importance of avoiding the temptation to collect as much data as possible; adequate time also needs to be set aside for data analysis and interpretation of what are often highly complex datasets.

Case study research has sometimes been criticised for lacking scientific rigour and providing little basis for generalisation (i.e. producing findings that may be transferable to other settings)[ 1 ]. There are several ways to address these concerns, including: the use of theoretical sampling (i.e. drawing on a particular conceptual framework); respondent validation (i.e. participants checking emerging findings and the researcher's interpretation, and providing an opinion as to whether they feel these are accurate); and transparency throughout the research process (see Table 8 )[ 8 , 18 – 21 , 23 , 26 ]. Transparency can be achieved by describing in detail the steps involved in case selection, data collection, the reasons for the particular methods chosen, and the researcher's background and level of involvement (i.e. being explicit about how the researcher has influenced data collection and interpretation). Seeking potential, alternative explanations, and being explicit about how interpretations and conclusions were reached, help readers to judge the trustworthiness of the case study report. Stake provides a critique checklist for a case study report (Table 9 )[ 8 ].

Conclusions

The case study approach allows, amongst other things, critical events, interventions, policy developments and programme-based service reforms to be studied in detail in a real-life context. It should therefore be considered when an experimental design is either inappropriate to answer the research questions posed or impossible to undertake. Considering the frequency with which implementations of innovations are now taking place in healthcare settings and how well the case study approach lends itself to in-depth, complex health service research, we believe this approach should be more widely considered by researchers. Though inherently challenging, the research case study can, if carefully conceptualised and thoughtfully undertaken and reported, yield powerful insights into many important aspects of health and healthcare delivery.

Yin RK: Case study research, design and method. 2009, London: Sage Publications Ltd., 4

Google Scholar  

Keen J, Packwood T: Qualitative research; case study evaluation. BMJ. 1995, 311: 444-446.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Sheikh A, Halani L, Bhopal R, Netuveli G, Partridge M, Car J, et al: Facilitating the Recruitment of Minority Ethnic People into Research: Qualitative Case Study of South Asians and Asthma. PLoS Med. 2009, 6 (10): 1-11.

Article   Google Scholar  

Pinnock H, Huby G, Powell A, Kielmann T, Price D, Williams S, et al: The process of planning, development and implementation of a General Practitioner with a Special Interest service in Primary Care Organisations in England and Wales: a comparative prospective case study. Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO). 2008, [ http://www.sdo.nihr.ac.uk/files/project/99-final-report.pdf ]

Robertson A, Cresswell K, Takian A, Petrakaki D, Crowe S, Cornford T, et al: Prospective evaluation of the implementation and adoption of NHS Connecting for Health's national electronic health record in secondary care in England: interim findings. BMJ. 2010, 41: c4564-

Pearson P, Steven A, Howe A, Sheikh A, Ashcroft D, Smith P, the Patient Safety Education Study Group: Learning about patient safety: organisational context and culture in the education of healthcare professionals. J Health Serv Res Policy. 2010, 15: 4-10. 10.1258/jhsrp.2009.009052.

Article   PubMed   Google Scholar  

van Harten WH, Casparie TF, Fisscher OA: The evaluation of the introduction of a quality management system: a process-oriented case study in a large rehabilitation hospital. Health Policy. 2002, 60 (1): 17-37. 10.1016/S0168-8510(01)00187-7.

Stake RE: The art of case study research. 1995, London: Sage Publications Ltd.

Sheikh A, Smeeth L, Ashcroft R: Randomised controlled trials in primary care: scope and application. Br J Gen Pract. 2002, 52 (482): 746-51.

PubMed   PubMed Central   Google Scholar  

King G, Keohane R, Verba S: Designing Social Inquiry. 1996, Princeton: Princeton University Press

Doolin B: Information technology as disciplinary technology: being critical in interpretative research on information systems. Journal of Information Technology. 1998, 13: 301-311. 10.1057/jit.1998.8.

George AL, Bennett A: Case studies and theory development in the social sciences. 2005, Cambridge, MA: MIT Press

Eccles M, the Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG): Designing theoretically-informed implementation interventions. Implementation Science. 2006, 1: 1-8. 10.1186/1748-5908-1-1.

Article   PubMed Central   Google Scholar  

Netuveli G, Hurwitz B, Levy M, Fletcher M, Barnes G, Durham SR, Sheikh A: Ethnic variations in UK asthma frequency, morbidity, and health-service use: a systematic review and meta-analysis. Lancet. 2005, 365 (9456): 312-7.

Sheikh A, Panesar SS, Lasserson T, Netuveli G: Recruitment of ethnic minorities to asthma studies. Thorax. 2004, 59 (7): 634-

CAS   PubMed   PubMed Central   Google Scholar  

Hellström I, Nolan M, Lundh U: 'We do things together': A case study of 'couplehood' in dementia. Dementia. 2005, 4: 7-22. 10.1177/1471301205049188.

Som CV: Nothing seems to have changed, nothing seems to be changing and perhaps nothing will change in the NHS: doctors' response to clinical governance. International Journal of Public Sector Management. 2005, 18: 463-477. 10.1108/09513550510608903.

Lincoln Y, Guba E: Naturalistic inquiry. 1985, Newbury Park: Sage Publications

Barbour RS: Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?. BMJ. 2001, 322: 1115-1117. 10.1136/bmj.322.7294.1115.

Mays N, Pope C: Qualitative research in health care: Assessing quality in qualitative research. BMJ. 2000, 320: 50-52. 10.1136/bmj.320.7226.50.

Mason J: Qualitative researching. 2002, London: Sage

Brazier A, Cooke K, Moravan V: Using Mixed Methods for Evaluating an Integrative Approach to Cancer Care: A Case Study. Integr Cancer Ther. 2008, 7: 5-17. 10.1177/1534735407313395.

Miles MB, Huberman M: Qualitative data analysis: an expanded sourcebook. 1994, CA: Sage Publications Inc., 2

Pope C, Ziebland S, Mays N: Analysing qualitative data. Qualitative research in health care. BMJ. 2000, 320: 114-116. 10.1136/bmj.320.7227.114.

Cresswell KM, Worth A, Sheikh A: Actor-Network Theory and its role in understanding the implementation of information technology developments in healthcare. BMC Med Inform Decis Mak. 2010, 10 (1): 67-10.1186/1472-6947-10-67.

Article   PubMed   PubMed Central   Google Scholar  

Malterud K: Qualitative research: standards, challenges, and guidelines. Lancet. 2001, 358: 483-488. 10.1016/S0140-6736(01)05627-6.

Article   CAS   PubMed   Google Scholar  

Yin R: Case study research: design and methods. 1994, Thousand Oaks, CA: Sage Publishing, 2

Yin R: Enhancing the quality of case studies in health services research. Health Serv Res. 1999, 34: 1209-1224.

Green J, Thorogood N: Qualitative methods for health research. 2009, Los Angeles: Sage, 2

Howcroft D, Trauth E: Handbook of Critical Information Systems Research, Theory and Application. 2005, Cheltenham, UK: Northampton, MA, USA: Edward Elgar

Book   Google Scholar  

Blakie N: Approaches to Social Enquiry. 1993, Cambridge: Polity Press

Doolin B: Power and resistance in the implementation of a medical management information system. Info Systems J. 2004, 14: 343-362. 10.1111/j.1365-2575.2004.00176.x.

Bloomfield BP, Best A: Management consultants: systems development, power and the translation of problems. Sociological Review. 1992, 40: 533-560.

Shanks G, Parr A: Positivist, single case study research in information systems: A critical analysis. Proceedings of the European Conference on Information Systems. 2003, Naples

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/11/100/prepub

Download references

Acknowledgements

We are grateful to the participants and colleagues who contributed to the individual case studies that we have drawn on. This work received no direct funding, but it has been informed by projects funded by Asthma UK, the NHS Service Delivery Organisation, NHS Connecting for Health Evaluation Programme, and Patient Safety Research Portfolio. We would also like to thank the expert reviewers for their insightful and constructive feedback. Our thanks are also due to Dr. Allison Worth who commented on an earlier draft of this manuscript.

Author information

Authors and affiliations.

Division of Primary Care, The University of Nottingham, Nottingham, UK

Sarah Crowe & Anthony Avery

Centre for Population Health Sciences, The University of Edinburgh, Edinburgh, UK

Kathrin Cresswell, Ann Robertson & Aziz Sheikh

School of Health in Social Science, The University of Edinburgh, Edinburgh, UK

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sarah Crowe .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors' contributions

AS conceived this article. SC, KC and AR wrote this paper with GH, AA and AS all commenting on various drafts. SC and AS are guarantors.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Crowe, S., Cresswell, K., Robertson, A. et al. The case study approach. BMC Med Res Methodol 11 , 100 (2011). https://doi.org/10.1186/1471-2288-11-100

Download citation

Received : 29 November 2010

Accepted : 27 June 2011

Published : 27 June 2011

DOI : https://doi.org/10.1186/1471-2288-11-100

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Case Study Approach
  • Electronic Health Record System
  • Case Study Design
  • Case Study Site
  • Case Study Report

BMC Medical Research Methodology

ISSN: 1471-2288

limitations of a qualitative case study

  • Open access
  • Published: 06 April 2024

Tools for assessing the methodological limitations of a QES—a short note

  • Heid Nøkleby   ORCID: orcid.org/0009-0008-9844-7279 1 ,
  • Heather Melanie R. Ames 1 ,
  • Lars Jørun Langøien 1 &
  • Christine Hillestad Hestevik 1  

Systematic Reviews volume  13 , Article number:  103 ( 2024 ) Cite this article

1 Altmetric

Metrics details

The increasing prevalence and application of qualitative evidence syntheses (QES) in decision-making processes underscore the need for robust tools to assess the methodological limitations of a completed QES. This commentary discusses the limitations of three existing tools and presents the authors’ efforts to address this gap. Through a simple comparative analysis, the three tools are examined in terms of their coverage of essential topic areas. The examination finds that existing assessment tools lack comprehensive coverage, clarity, and grounding in qualitative research principles. The authors advocate for the development of a new collaboratively developed evidence-based tool rooted in qualitative methodology and best practice methods. The conclusion emphasizes the necessity of a tool that can provide a comprehensive judgement on the methodological limitations of a QES, addressing the needs of end-users, and ultimately enhancing the trustworthiness of QES findings in decision-making processes.

Peer Review reports

As qualitative evidence syntheses (QES) are becoming more common and increasingly used in decision-making processes [ 1 , 2 , 3 , 4 , 5 ], there is a need for a tool to assess the methodological limitations of a complete QES. This methodological assessment tool could help users to understand the trust they can place in the findings of a QES and help to interpret further use. In our work, this type of assessment tool would primarily be useful when an existing QES is found that answers a commissioner’s question. In this case, we need to be able to assess the methodological limitations of the completed QES to make a judgement for the commissioner on the extent to which the findings can be trusted and used to suit their purposes. We refer to an assessment tool, and not a checklist, as a deeper methodological understanding of the limitations of a QES is needed to assess the synthesis and how its methodological limitations impact on further use. We believe that the scoring or ranking which are the products of a checklist would not allow for a deep enough evaluation of and reflection around the methodological limitations of the QES and how they relate to the context and question that the QES is going to be used in and for.

The foundation for the discussion in this commentary was a teaching experience the team had in 2022. A request for course content was how to assess the methodological limitations of a QES. We recently had an in-house discussion about the three tools we had identified as options for assessing the methodological limitations of a QES. All three tools are in beta or preliminary versions. We discovered that the assessment tools are not easily accessible. Knowledge of their existence and whereabouts is necessary to locate them. All tools have been developed to meet an internal need (fit for purpose). None of them have been developed through best practice methods [ 6 , 7 ]. The three tools are:

Tool 1 : Criteria for assessing how well a qualitative evidence syntheses (systematic reviews of qualitative studies) was conducted , a tool developed by Lewin and colleagues in 2012 [ 8 , 9 ]

Tool 2: a prototype assessment tool based on AMSTAR 2 [ 10 ], Measurement Appraisal Checklist to Assess Qualitative Evidence Syntheses (MACAQuES) by Booth and colleagues from 2019 [ 11 ]

Tool 3: Review template for qualitative evidence synthesis (QES) , developed by the Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU) based on the ENTREQ reporting guidance [ 12 ] in 2023, is published but is still marked as “under development” [ 13 ]

We wanted to expand our in-house discussion further for teaching purposes. To do this, first, the authors compared the QES methodological assessment tools in a table and through discussion. Next, we incorporated them into an introductory course on QES methods delivered in October 2022. During the course, we had students reflect over any topics or questions they felt were missing from the existing tools based on the course content. Finally, we reflected on the student feedback and our experiences to assess and conclude that none of the tools fully met our needs.

In this short note, we aim to briefly present and compare items across the three assessment tools we identified and describe what we believe to be their strengths and limitations.

Three assessment tools

We have compared the three QES assessment tools (see Table  1 ). An x was placed in the table if an item was mentioned in a question or a prompt.

Seven of the eighteen topic areas are covered in all three tools (review question, inclusion criteria, literature search, methodological assessment of the included studies, analysis/synthesis, findings and reflexivity). Four of the topic areas are covered by two tools; a description of the excluded studies is covered in tool 1 and tool 2. Planning/protocol, conflict of interest, and confidence in the findings are covered in tool 2 and tool 3. However, tool 1 was published before the use of GRADE CERQual was implemented, so it is not surprising that that topic area is missing. Six topic areas are covered only by one assessment tool; tool 2 asks users to think through patient involvement, the description of the included studies, data extraction/coding, and dissemination bias. Tool 3 asks users to reflect on researchers’ competence, screening, and other.

Tool 1 has considerably fewer topic areas but includes prompt questions to help the user think through the topic areas. Tool 2 also provides prompt questions or items of note that users should consider when thinking through the topic area. Tool 3 is accompanied by a user guide.

All three tools require experience with and knowledge of qualitative research. This knowledge is needed to interpret the items/questions in a “qualitative manner” to ensure that methodological limitations relevant to qualitative research are assessed. Many questions are not explicitly formulated, meaning that the end user needs to understand qualitative research principles and practices to interpret and apply them. For example, a detailed knowledge around searching and where relevant qualitative evidence is located [ 13 ], a knowledge of which synthesis method is appropriate for which type of question [ 8 , 9 , 11 , 13 ], and a knowledge of the QES authors background, experience, and competence [ 13 ]. Finally, the tools raise concepts that may be new to some researchers such as the concept of the impact of dissemination bias in primary qualitative research and its implications on QES findings [ 11 ].

Need for collaboration in developing a new evidence-based methodological assessment tool for QES

Based on our comparison of the three assessment tools, we think there is a need to systematically search for map and assess existing tools. If there is not an existing tool which has been developed in an evidence-based way, then a tool should be considered. Ideally, the end goal would be to develop a new assessment tool that is based on the principles of qualitative research and qualitative evidence syntheses using best practice methods for assessment tool development. The development of the new tool should follow best practice methods so that it reflects all items relevant for the assessment of a completed QES, is based on qualitative methodology, and addresses the needs of the end user—being able to assess the limitations of a completed QES.

This process should be a collaborative effort within the QES community. The first step would be a systematic search for existing tools and the identification of relevant principles in these tools. Additional principles should be gathered from focus groups. This exploratory step would be followed by a Delphi process where stakeholders could come to an agreement on the principles that should be included in a future tool. After the consensus process has been completed, an assessment tool could begin to be developed and user tested.

Recently, this process of collaboratively developing an evidence-based tool for the assessment of the methodological limitations of primary qualitative studies included in a QES (CAMELOT) has been completed [ 5 , 14 , 15 , 16 ]. The CAMELOT project followed the same process we describe above, involving a large number of relevant stakeholders in a collaborative process to determine what was important to include and how the tool could be used. CAMELOT, along with other previous [ 17 , 18 , 19 , 20 , 21 , 22 ] and ongoing [ 23 ] projects that have used the same methodology, lead us to believe that this process would lead to an evidence-based assessment tool for the assessment of the methodological limitations of a QES. We believe that the development of a tool for assessing the methodological limitations of a qualitative evidence synthesis is needed.

In conclusion, we believe that none of the QES methodological assessment tools covered all of the areas that were raised by students as well as our reflections from working in the field. We found that the tools did not seem to be clearly grounded in qualitative research methods (for example words or expressions common in quantitative research were used). We also found that they could not provide a comprehensive/complete judgement on the methodological limitations of a QES that we could present to a commissioner or use to make a decision as critical areas or items were missing that we feel should be considered. We believe that the development of a tool for assessing the methodological limitations of a qualitative evidence synthesis is needed.

Availability of data and materials

Not applicable.

Lewin S, Glenton C, Lawrie TA, Downe S, Finlayson KW, Rosenbaum S, et al. Qualitative evidence synthesis (QES) for guidelines: paper 2–using qualitative evidence synthesis findings to inform evidence-to-decision frameworks and recommendations. Health Res Policy Syst. 2019;17(1):1–18.

Article   Google Scholar  

Downe S, Finlayson KW, Lawrie TA, Lewin SA, Glenton C, Rosenbaum S, et al. Qualitative evidence synthesis (QES) for guidelines: paper 1–using qualitative evidence synthesis to inform guideline scope and develop qualitative findings statements. Health Res Policy Syst. 2019;17(1):1–12.

Glenton C, Lewin S, Lawrie TA, Barreix M, Downe S, Finlayson KW, et al. Qualitative evidence synthesis (QES) for guidelines: paper 3–using qualitative evidence syntheses to develop implementation considerations and inform implementation processes. Health Res Policy Syst. 2019;17(1):1–14.

Flemming K, Noyes J. Qualitative evidence synthesis: where are we at? Int J Qual Methods. 2021;20:1609406921993276.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):1–13.

Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2):e1000217.

Article   PubMed   PubMed Central   Google Scholar  

Whiting P, Wolff R, Mallett S, Simera I, Savović J. A proposed framework for developing quality assessment tools. Syst Rev. 2017;6:1–9.

Lewin S. Criteria for assessing how well a qualitative evidence syntheses (systematic reviews of qualitative studies) was conducted; EPOC resources for review authors. Oslo: Norwegian Institute of Public Health; 2018.

Google Scholar  

Lewin S, Bosch-Capblanch X, Oliver S, Akl EA, Vist GE, Lavis JN, et al. Guidance for evidence-informed policies about health systems: assessing how much confidence to place in the research evidence. PLoS Med. 2012;9(3):e1001187.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.

Booth AftCQaIMG. Prototype - Measurement Appraisal Checklist to Assess Qualitative Evidence Syntheses (QES) (MACAQuES). Sheffield: University of Sheffield; 2019.

Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12(1):1–8.

SBU. Granskningsmall för kvalitativa evidenssynteser (QES) (Tool to assess methodological limitations of qualitative evidence synthesis). Stockholm: SBU; 2023.

Munthe-Kaas H, Bohren MA, Glenton C, Lewin S, Noyes J, Tunçalp Ö, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 3: how to assess methodological limitations. Implement Sci. 2018;13(1):25–32.

Munthe-Kaas HM, Sommer I, Noyes J, Cooper S, Garside R, Hannes K, et al. Development of the CAMELOT approach for considering methodological limitations of qualitative research in the context of GRADE-CERQual and qualitative evidence syntheses – protocol (version 1). Geneve: Zenodo; 2023.

Munthe-Kaas AHB, Sommer I, Cooper S, Garside R, Hannes K, Noyes J. Developing CAMELOT for assessing methodological limitations of qualitative research for inclusion in qualitative evidence syntheses. Submitted to Cochrane Evidence Synthesis and Methods. 2024.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2020;2021:372.

Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;372:n160.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement. J Clin Epidemiol. 2021;134:103–12.

Article   PubMed   Google Scholar  

France E, Ring N, Noyes J, Maxwell M, Jepson R, Duncan E, et al. Protocol-developing meta-ethnography reporting guidelines (eMERGe). BMC Med Res Methodol. 2015;15:1–14.

France EF, Cunningham M, Ring N, Uny I, Duncan EA, Jepson RG, et al. Improving reporting of meta-ethnography: the eMERGe reporting guidance. BMC Med Res Methodol. 2019;19(1):1–13.

Cunningham M, France EF, Ring N, Uny I, Duncan EA, Roberts RJ, et al. Developing a reporting guideline to improve meta-ethnography in health research: the eMERGe mixed-methods study. Health Serv Deliv Res. 2019;7(4):1–116.

Svendsen C, Whaley P, Vist GE, Husøy T, Beronius A, Di Consiglio E, et al. Protocol for designing INVITES-IN, a tool for assessing the internal validity of in vitro studies. Evid Based Toxicol. 2023;1(1):2232415.

Download references

Acknowledgements

Thanks to the authors of the tools for sharing information.

This work was funded by the Norwegian Institute of Public Health.

Author information

Authors and affiliations.

Norwegian Institute of Public Health, Oslo, Norway

Heid Nøkleby, Heather Melanie R. Ames, Lars Jørun Langøien & Christine Hillestad Hestevik

You can also search for this author in PubMed   Google Scholar

Contributions

All four authors (HN, HA, LJL, CH) participated in planning, giving, or evaluating the initial teaching experience. HA had the idea to the commentary. HN made the comparison between the three assessment tools and sketched the text, and HA, LJL, and CH revised it. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Heid Nøkleby .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Nøkleby, H., Ames, H.M.R., Langøien, L.J. et al. Tools for assessing the methodological limitations of a QES—a short note. Syst Rev 13 , 103 (2024). https://doi.org/10.1186/s13643-024-02511-6

Download citation

Received : 10 January 2024

Accepted : 07 March 2024

Published : 06 April 2024

DOI : https://doi.org/10.1186/s13643-024-02511-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Qualitative evidence synthesis
  • Assessment tool
  • Methodological limitations

Systematic Reviews

ISSN: 2046-4053

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

limitations of a qualitative case study

  • USC Libraries
  • Research Guides

Organizing Your Social Sciences Research Paper

  • Limitations of the Study
  • Purpose of Guide
  • Design Flaws to Avoid
  • Independent and Dependent Variables
  • Glossary of Research Terms
  • Reading Research Effectively
  • Narrowing a Topic Idea
  • Broadening a Topic Idea
  • Extending the Timeliness of a Topic Idea
  • Academic Writing Style
  • Choosing a Title
  • Making an Outline
  • Paragraph Development
  • Research Process Video Series
  • Executive Summary
  • The C.A.R.S. Model
  • Background Information
  • The Research Problem/Question
  • Theoretical Framework
  • Citation Tracking
  • Content Alert Services
  • Evaluating Sources
  • Primary Sources
  • Secondary Sources
  • Tiertiary Sources
  • Scholarly vs. Popular Publications
  • Qualitative Methods
  • Quantitative Methods
  • Insiderness
  • Using Non-Textual Elements
  • Common Grammar Mistakes
  • Writing Concisely
  • Avoiding Plagiarism
  • Footnotes or Endnotes?
  • Further Readings
  • Generative AI and Writing
  • USC Libraries Tutorials and Other Guides
  • Bibliography

The limitations of the study are those characteristics of design or methodology that impacted or influenced the interpretation of the findings from your research. Study limitations are the constraints placed on the ability to generalize from the results, to further describe applications to practice, and/or related to the utility of findings that are the result of the ways in which you initially chose to design the study or the method used to establish internal and external validity or the result of unanticipated challenges that emerged during the study.

Price, James H. and Judy Murnan. “Research Limitations and the Necessity of Reporting Them.” American Journal of Health Education 35 (2004): 66-67; Theofanidis, Dimitrios and Antigoni Fountouki. "Limitations and Delimitations in the Research Process." Perioperative Nursing 7 (September-December 2018): 155-163. .

Importance of...

Always acknowledge a study's limitations. It is far better that you identify and acknowledge your study’s limitations than to have them pointed out by your professor and have your grade lowered because you appeared to have ignored them or didn't realize they existed.

Keep in mind that acknowledgment of a study's limitations is an opportunity to make suggestions for further research. If you do connect your study's limitations to suggestions for further research, be sure to explain the ways in which these unanswered questions may become more focused because of your study.

Acknowledgment of a study's limitations also provides you with opportunities to demonstrate that you have thought critically about the research problem, understood the relevant literature published about it, and correctly assessed the methods chosen for studying the problem. A key objective of the research process is not only discovering new knowledge but also to confront assumptions and explore what we don't know.

Claiming limitations is a subjective process because you must evaluate the impact of those limitations . Don't just list key weaknesses and the magnitude of a study's limitations. To do so diminishes the validity of your research because it leaves the reader wondering whether, or in what ways, limitation(s) in your study may have impacted the results and conclusions. Limitations require a critical, overall appraisal and interpretation of their impact. You should answer the question: do these problems with errors, methods, validity, etc. eventually matter and, if so, to what extent?

Price, James H. and Judy Murnan. “Research Limitations and the Necessity of Reporting Them.” American Journal of Health Education 35 (2004): 66-67; Structure: How to Structure the Research Limitations Section of Your Dissertation. Dissertations and Theses: An Online Textbook. Laerd.com.

Descriptions of Possible Limitations

All studies have limitations . However, it is important that you restrict your discussion to limitations related to the research problem under investigation. For example, if a meta-analysis of existing literature is not a stated purpose of your research, it should not be discussed as a limitation. Do not apologize for not addressing issues that you did not promise to investigate in the introduction of your paper.

Here are examples of limitations related to methodology and the research process you may need to describe and discuss how they possibly impacted your results. Note that descriptions of limitations should be stated in the past tense because they were discovered after you completed your research.

Possible Methodological Limitations

  • Sample size -- the number of the units of analysis you use in your study is dictated by the type of research problem you are investigating. Note that, if your sample size is too small, it will be difficult to find significant relationships from the data, as statistical tests normally require a larger sample size to ensure a representative distribution of the population and to be considered representative of groups of people to whom results will be generalized or transferred. Note that sample size is generally less relevant in qualitative research if explained in the context of the research problem.
  • Lack of available and/or reliable data -- a lack of data or of reliable data will likely require you to limit the scope of your analysis, the size of your sample, or it can be a significant obstacle in finding a trend and a meaningful relationship. You need to not only describe these limitations but provide cogent reasons why you believe data is missing or is unreliable. However, don’t just throw up your hands in frustration; use this as an opportunity to describe a need for future research based on designing a different method for gathering data.
  • Lack of prior research studies on the topic -- citing prior research studies forms the basis of your literature review and helps lay a foundation for understanding the research problem you are investigating. Depending on the currency or scope of your research topic, there may be little, if any, prior research on your topic. Before assuming this to be true, though, consult with a librarian! In cases when a librarian has confirmed that there is little or no prior research, you may be required to develop an entirely new research typology [for example, using an exploratory rather than an explanatory research design ]. Note again that discovering a limitation can serve as an important opportunity to identify new gaps in the literature and to describe the need for further research.
  • Measure used to collect the data -- sometimes it is the case that, after completing your interpretation of the findings, you discover that the way in which you gathered data inhibited your ability to conduct a thorough analysis of the results. For example, you regret not including a specific question in a survey that, in retrospect, could have helped address a particular issue that emerged later in the study. Acknowledge the deficiency by stating a need for future researchers to revise the specific method for gathering data.
  • Self-reported data -- whether you are relying on pre-existing data or you are conducting a qualitative research study and gathering the data yourself, self-reported data is limited by the fact that it rarely can be independently verified. In other words, you have to the accuracy of what people say, whether in interviews, focus groups, or on questionnaires, at face value. However, self-reported data can contain several potential sources of bias that you should be alert to and note as limitations. These biases become apparent if they are incongruent with data from other sources. These are: (1) selective memory [remembering or not remembering experiences or events that occurred at some point in the past]; (2) telescoping [recalling events that occurred at one time as if they occurred at another time]; (3) attribution [the act of attributing positive events and outcomes to one's own agency, but attributing negative events and outcomes to external forces]; and, (4) exaggeration [the act of representing outcomes or embellishing events as more significant than is actually suggested from other data].

Possible Limitations of the Researcher

  • Access -- if your study depends on having access to people, organizations, data, or documents and, for whatever reason, access is denied or limited in some way, the reasons for this needs to be described. Also, include an explanation why being denied or limited access did not prevent you from following through on your study.
  • Longitudinal effects -- unlike your professor, who can literally devote years [even a lifetime] to studying a single topic, the time available to investigate a research problem and to measure change or stability over time is constrained by the due date of your assignment. Be sure to choose a research problem that does not require an excessive amount of time to complete the literature review, apply the methodology, and gather and interpret the results. If you're unsure whether you can complete your research within the confines of the assignment's due date, talk to your professor.
  • Cultural and other type of bias -- we all have biases, whether we are conscience of them or not. Bias is when a person, place, event, or thing is viewed or shown in a consistently inaccurate way. Bias is usually negative, though one can have a positive bias as well, especially if that bias reflects your reliance on research that only support your hypothesis. When proof-reading your paper, be especially critical in reviewing how you have stated a problem, selected the data to be studied, what may have been omitted, the manner in which you have ordered events, people, or places, how you have chosen to represent a person, place, or thing, to name a phenomenon, or to use possible words with a positive or negative connotation. NOTE :   If you detect bias in prior research, it must be acknowledged and you should explain what measures were taken to avoid perpetuating that bias. For example, if a previous study only used boys to examine how music education supports effective math skills, describe how your research expands the study to include girls.
  • Fluency in a language -- if your research focuses , for example, on measuring the perceived value of after-school tutoring among Mexican-American ESL [English as a Second Language] students and you are not fluent in Spanish, you are limited in being able to read and interpret Spanish language research studies on the topic or to speak with these students in their primary language. This deficiency should be acknowledged.

Aguinis, Hermam and Jeffrey R. Edwards. “Methodological Wishes for the Next Decade and How to Make Wishes Come True.” Journal of Management Studies 51 (January 2014): 143-174; Brutus, Stéphane et al. "Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations." Journal of Management 39 (January 2013): 48-75; Senunyeme, Emmanuel K. Business Research Methods. Powerpoint Presentation. Regent University of Science and Technology; ter Riet, Gerben et al. “All That Glitters Isn't Gold: A Survey on Acknowledgment of Limitations in Biomedical Studies.” PLOS One 8 (November 2013): 1-6.

Structure and Writing Style

Information about the limitations of your study are generally placed either at the beginning of the discussion section of your paper so the reader knows and understands the limitations before reading the rest of your analysis of the findings, or, the limitations are outlined at the conclusion of the discussion section as an acknowledgement of the need for further study. Statements about a study's limitations should not be buried in the body [middle] of the discussion section unless a limitation is specific to something covered in that part of the paper. If this is the case, though, the limitation should be reiterated at the conclusion of the section.

If you determine that your study is seriously flawed due to important limitations , such as, an inability to acquire critical data, consider reframing it as an exploratory study intended to lay the groundwork for a more complete research study in the future. Be sure, though, to specifically explain the ways that these flaws can be successfully overcome in a new study.

But, do not use this as an excuse for not developing a thorough research paper! Review the tab in this guide for developing a research topic . If serious limitations exist, it generally indicates a likelihood that your research problem is too narrowly defined or that the issue or event under study is too recent and, thus, very little research has been written about it. If serious limitations do emerge, consult with your professor about possible ways to overcome them or how to revise your study.

When discussing the limitations of your research, be sure to:

  • Describe each limitation in detailed but concise terms;
  • Explain why each limitation exists;
  • Provide the reasons why each limitation could not be overcome using the method(s) chosen to acquire or gather the data [cite to other studies that had similar problems when possible];
  • Assess the impact of each limitation in relation to the overall findings and conclusions of your study; and,
  • If appropriate, describe how these limitations could point to the need for further research.

Remember that the method you chose may be the source of a significant limitation that has emerged during your interpretation of the results [for example, you didn't interview a group of people that you later wish you had]. If this is the case, don't panic. Acknowledge it, and explain how applying a different or more robust methodology might address the research problem more effectively in a future study. A underlying goal of scholarly research is not only to show what works, but to demonstrate what doesn't work or what needs further clarification.

Aguinis, Hermam and Jeffrey R. Edwards. “Methodological Wishes for the Next Decade and How to Make Wishes Come True.” Journal of Management Studies 51 (January 2014): 143-174; Brutus, Stéphane et al. "Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations." Journal of Management 39 (January 2013): 48-75; Ioannidis, John P.A. "Limitations are not Properly Acknowledged in the Scientific Literature." Journal of Clinical Epidemiology 60 (2007): 324-329; Pasek, Josh. Writing the Empirical Social Science Research Paper: A Guide for the Perplexed. January 24, 2012. Academia.edu; Structure: How to Structure the Research Limitations Section of Your Dissertation. Dissertations and Theses: An Online Textbook. Laerd.com; What Is an Academic Paper? Institute for Writing Rhetoric. Dartmouth College; Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University.

Writing Tip

Don't Inflate the Importance of Your Findings!

After all the hard work and long hours devoted to writing your research paper, it is easy to get carried away with attributing unwarranted importance to what you’ve done. We all want our academic work to be viewed as excellent and worthy of a good grade, but it is important that you understand and openly acknowledge the limitations of your study. Inflating the importance of your study's findings could be perceived by your readers as an attempt hide its flaws or encourage a biased interpretation of the results. A small measure of humility goes a long way!

Another Writing Tip

Negative Results are Not a Limitation!

Negative evidence refers to findings that unexpectedly challenge rather than support your hypothesis. If you didn't get the results you anticipated, it may mean your hypothesis was incorrect and needs to be reformulated. Or, perhaps you have stumbled onto something unexpected that warrants further study. Moreover, the absence of an effect may be very telling in many situations, particularly in experimental research designs. In any case, your results may very well be of importance to others even though they did not support your hypothesis. Do not fall into the trap of thinking that results contrary to what you expected is a limitation to your study. If you carried out the research well, they are simply your results and only require additional interpretation.

Lewis, George H. and Jonathan F. Lewis. “The Dog in the Night-Time: Negative Evidence in Social Research.” The British Journal of Sociology 31 (December 1980): 544-558.

Yet Another Writing Tip

Sample Size Limitations in Qualitative Research

Sample sizes are typically smaller in qualitative research because, as the study goes on, acquiring more data does not necessarily lead to more information. This is because one occurrence of a piece of data, or a code, is all that is necessary to ensure that it becomes part of the analysis framework. However, it remains true that sample sizes that are too small cannot adequately support claims of having achieved valid conclusions and sample sizes that are too large do not permit the deep, naturalistic, and inductive analysis that defines qualitative inquiry. Determining adequate sample size in qualitative research is ultimately a matter of judgment and experience in evaluating the quality of the information collected against the uses to which it will be applied and the particular research method and purposeful sampling strategy employed. If the sample size is found to be a limitation, it may reflect your judgment about the methodological technique chosen [e.g., single life history study versus focus group interviews] rather than the number of respondents used.

Boddy, Clive Roland. "Sample Size for Qualitative Research." Qualitative Market Research: An International Journal 19 (2016): 426-432; Huberman, A. Michael and Matthew B. Miles. "Data Management and Analysis Methods." In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 428-444; Blaikie, Norman. "Confounding Issues Related to Determining Sample Size in Qualitative Research." International Journal of Social Research Methodology 21 (2018): 635-641; Oppong, Steward Harrison. "The Problem of Sampling in qualitative Research." Asian Journal of Management Sciences and Education 2 (2013): 202-210.

  • << Previous: 8. The Discussion
  • Next: 9. The Conclusion >>
  • Last Updated: Apr 5, 2024 1:38 PM
  • URL: https://libguides.usc.edu/writingguide

How to Write Limitations of the Study (with examples)

This blog emphasizes the importance of recognizing and effectively writing about limitations in research. It discusses the types of limitations, their significance, and provides guidelines for writing about them, highlighting their role in advancing scholarly research.

Updated on August 24, 2023

a group of researchers writing their limitation of their study

No matter how well thought out, every research endeavor encounters challenges. There is simply no way to predict all possible variances throughout the process.

These uncharted boundaries and abrupt constraints are known as limitations in research . Identifying and acknowledging limitations is crucial for conducting rigorous studies. Limitations provide context and shed light on gaps in the prevailing inquiry and literature.

This article explores the importance of recognizing limitations and discusses how to write them effectively. By interpreting limitations in research and considering prevalent examples, we aim to reframe the perception from shameful mistakes to respectable revelations.

What are limitations in research?

In the clearest terms, research limitations are the practical or theoretical shortcomings of a study that are often outside of the researcher’s control . While these weaknesses limit the generalizability of a study’s conclusions, they also present a foundation for future research.

Sometimes limitations arise from tangible circumstances like time and funding constraints, or equipment and participant availability. Other times the rationale is more obscure and buried within the research design. Common types of limitations and their ramifications include:

  • Theoretical: limits the scope, depth, or applicability of a study.
  • Methodological: limits the quality, quantity, or diversity of the data.
  • Empirical: limits the representativeness, validity, or reliability of the data.
  • Analytical: limits the accuracy, completeness, or significance of the findings.
  • Ethical: limits the access, consent, or confidentiality of the data.

Regardless of how, when, or why they arise, limitations are a natural part of the research process and should never be ignored . Like all other aspects, they are vital in their own purpose.

Why is identifying limitations important?

Whether to seek acceptance or avoid struggle, humans often instinctively hide flaws and mistakes. Merging this thought process into research by attempting to hide limitations, however, is a bad idea. It has the potential to negate the validity of outcomes and damage the reputation of scholars.

By identifying and addressing limitations throughout a project, researchers strengthen their arguments and curtail the chance of peer censure based on overlooked mistakes. Pointing out these flaws shows an understanding of variable limits and a scrupulous research process.

Showing awareness of and taking responsibility for a project’s boundaries and challenges validates the integrity and transparency of a researcher. It further demonstrates the researchers understand the applicable literature and have thoroughly evaluated their chosen research methods.

Presenting limitations also benefits the readers by providing context for research findings. It guides them to interpret the project’s conclusions only within the scope of very specific conditions. By allowing for an appropriate generalization of the findings that is accurately confined by research boundaries and is not too broad, limitations boost a study’s credibility .

Limitations are true assets to the research process. They highlight opportunities for future research. When researchers identify the limitations of their particular approach to a study question, they enable precise transferability and improve chances for reproducibility. 

Simply stating a project’s limitations is not adequate for spurring further research, though. To spark the interest of other researchers, these acknowledgements must come with thorough explanations regarding how the limitations affected the current study and how they can potentially be overcome with amended methods.

How to write limitations

Typically, the information about a study’s limitations is situated either at the beginning of the discussion section to provide context for readers or at the conclusion of the discussion section to acknowledge the need for further research. However, it varies depending upon the target journal or publication guidelines. 

Don’t hide your limitations

It is also important to not bury a limitation in the body of the paper unless it has a unique connection to a topic in that section. If so, it needs to be reiterated with the other limitations or at the conclusion of the discussion section. Wherever it is included in the manuscript, ensure that the limitations section is prominently positioned and clearly introduced.

While maintaining transparency by disclosing limitations means taking a comprehensive approach, it is not necessary to discuss everything that could have potentially gone wrong during the research study. If there is no commitment to investigation in the introduction, it is unnecessary to consider the issue a limitation to the research. Wholly consider the term ‘limitations’ and ask, “Did it significantly change or limit the possible outcomes?” Then, qualify the occurrence as either a limitation to include in the current manuscript or as an idea to note for other projects. 

Writing limitations

Once the limitations are concretely identified and it is decided where they will be included in the paper, researchers are ready for the writing task. Including only what is pertinent, keeping explanations detailed but concise, and employing the following guidelines is key for crafting valuable limitations:

1) Identify and describe the limitations : Clearly introduce the limitation by classifying its form and specifying its origin. For example:

  • An unintentional bias encountered during data collection
  • An intentional use of unplanned post-hoc data analysis

2) Explain the implications : Describe how the limitation potentially influences the study’s findings and how the validity and generalizability are subsequently impacted. Provide examples and evidence to support claims of the limitations’ effects without making excuses or exaggerating their impact. Overall, be transparent and objective in presenting the limitations, without undermining the significance of the research. 

3) Provide alternative approaches for future studies : Offer specific suggestions for potential improvements or avenues for further investigation. Demonstrate a proactive approach by encouraging future research that addresses the identified gaps and, therefore, expands the knowledge base.

Whether presenting limitations as an individual section within the manuscript or as a subtopic in the discussion area, authors should use clear headings and straightforward language to facilitate readability. There is no need to complicate limitations with jargon, computations, or complex datasets.

Examples of common limitations

Limitations are generally grouped into two categories , methodology and research process .

Methodology limitations

Methodology may include limitations due to:

  • Sample size
  • Lack of available or reliable data
  • Lack of prior research studies on the topic
  • Measure used to collect the data
  • Self-reported data

methodology limitation example

The researcher is addressing how the large sample size requires a reassessment of the measures used to collect and analyze the data.

Research process limitations

Limitations during the research process may arise from:

  • Access to information
  • Longitudinal effects
  • Cultural and other biases
  • Language fluency
  • Time constraints

research process limitations example

The author is pointing out that the model’s estimates are based on potentially biased observational studies.

Final thoughts

Successfully proving theories and touting great achievements are only two very narrow goals of scholarly research. The true passion and greatest efforts of researchers comes more in the form of confronting assumptions and exploring the obscure.

In many ways, recognizing and sharing the limitations of a research study both allows for and encourages this type of discovery that continuously pushes research forward. By using limitations to provide a transparent account of the project's boundaries and to contextualize the findings, researchers pave the way for even more robust and impactful research in the future.

Charla Viera, MS

See our "Privacy Policy"

Ensure your structure and ideas are consistent and clearly communicated

Pair your Premium Editing with our add-on service Presubmission Review for an overall assessment of your manuscript.

helpful professor logo

10 Case Study Advantages and Disadvantages

case study advantages and disadvantages, explained below

A case study in academic research is a detailed and in-depth examination of a specific instance or event, generally conducted through a qualitative approach to data.

The most common case study definition that I come across is is Robert K. Yin’s (2003, p. 13) quote provided below:

“An empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident.”

Researchers conduct case studies for a number of reasons, such as to explore complex phenomena within their real-life context, to look at a particularly interesting instance of a situation, or to dig deeper into something of interest identified in a wider-scale project.

While case studies render extremely interesting data, they have many limitations and are not suitable for all studies. One key limitation is that a case study’s findings are not usually generalizable to broader populations because one instance cannot be used to infer trends across populations.

Case Study Advantages and Disadvantages

1. in-depth analysis of complex phenomena.

Case study design allows researchers to delve deeply into intricate issues and situations.

By focusing on a specific instance or event, researchers can uncover nuanced details and layers of understanding that might be missed with other research methods, especially large-scale survey studies.

As Lee and Saunders (2017) argue,

“It allows that particular event to be studies in detail so that its unique qualities may be identified.”

This depth of analysis can provide rich insights into the underlying factors and dynamics of the studied phenomenon.

2. Holistic Understanding

Building on the above point, case studies can help us to understand a topic holistically and from multiple angles.

This means the researcher isn’t restricted to just examining a topic by using a pre-determined set of questions, as with questionnaires. Instead, researchers can use qualitative methods to delve into the many different angles, perspectives, and contextual factors related to the case study.

We can turn to Lee and Saunders (2017) again, who notes that case study researchers “develop a deep, holistic understanding of a particular phenomenon” with the intent of deeply understanding the phenomenon.

3. Examination of rare and Unusual Phenomena

We need to use case study methods when we stumble upon “rare and unusual” (Lee & Saunders, 2017) phenomena that would tend to be seen as mere outliers in population studies.

Take, for example, a child genius. A population study of all children of that child’s age would merely see this child as an outlier in the dataset, and this child may even be removed in order to predict overall trends.

So, to truly come to an understanding of this child and get insights into the environmental conditions that led to this child’s remarkable cognitive development, we need to do an in-depth study of this child specifically – so, we’d use a case study.

4. Helps Reveal the Experiences of Marginalzied Groups

Just as rare and unsual cases can be overlooked in population studies, so too can the experiences, beliefs, and perspectives of marginalized groups.

As Lee and Saunders (2017) argue, “case studies are also extremely useful in helping the expression of the voices of people whose interests are often ignored.”

Take, for example, the experiences of minority populations as they navigate healthcare systems. This was for many years a “hidden” phenomenon, not examined by researchers. It took case study designs to truly reveal this phenomenon, which helped to raise practitioners’ awareness of the importance of cultural sensitivity in medicine.

5. Ideal in Situations where Researchers cannot Control the Variables

Experimental designs – where a study takes place in a lab or controlled environment – are excellent for determining cause and effect . But not all studies can take place in controlled environments (Tetnowski, 2015).

When we’re out in the field doing observational studies or similar fieldwork, we don’t have the freedom to isolate dependent and independent variables. We need to use alternate methods.

Case studies are ideal in such situations.

A case study design will allow researchers to deeply immerse themselves in a setting (potentially combining it with methods such as ethnography or researcher observation) in order to see how phenomena take place in real-life settings.

6. Supports the generation of new theories or hypotheses

While large-scale quantitative studies such as cross-sectional designs and population surveys are excellent at testing theories and hypotheses on a large scale, they need a hypothesis to start off with!

This is where case studies – in the form of grounded research – come in. Often, a case study doesn’t start with a hypothesis. Instead, it ends with a hypothesis based upon the findings within a singular setting.

The deep analysis allows for hypotheses to emerge, which can then be taken to larger-scale studies in order to conduct further, more generalizable, testing of the hypothesis or theory.

7. Reveals the Unexpected

When a largescale quantitative research project has a clear hypothesis that it will test, it often becomes very rigid and has tunnel-vision on just exploring the hypothesis.

Of course, a structured scientific examination of the effects of specific interventions targeted at specific variables is extermely valuable.

But narrowly-focused studies often fail to shine a spotlight on unexpected and emergent data. Here, case studies come in very useful. Oftentimes, researchers set their eyes on a phenomenon and, when examining it closely with case studies, identify data and come to conclusions that are unprecedented, unforeseen, and outright surprising.

As Lars Meier (2009, p. 975) marvels, “where else can we become a part of foreign social worlds and have the chance to become aware of the unexpected?”

Disadvantages

1. not usually generalizable.

Case studies are not generalizable because they tend not to look at a broad enough corpus of data to be able to infer that there is a trend across a population.

As Yang (2022) argues, “by definition, case studies can make no claims to be typical.”

Case studies focus on one specific instance of a phenomenon. They explore the context, nuances, and situational factors that have come to bear on the case study. This is really useful for bringing to light important, new, and surprising information, as I’ve already covered.

But , it’s not often useful for generating data that has validity beyond the specific case study being examined.

2. Subjectivity in interpretation

Case studies usually (but not always) use qualitative data which helps to get deep into a topic and explain it in human terms, finding insights unattainable by quantitative data.

But qualitative data in case studies relies heavily on researcher interpretation. While researchers can be trained and work hard to focus on minimizing subjectivity (through methods like triangulation), it often emerges – some might argue it’s innevitable in qualitative studies.

So, a criticism of case studies could be that they’re more prone to subjectivity – and researchers need to take strides to address this in their studies.

3. Difficulty in replicating results

Case study research is often non-replicable because the study takes place in complex real-world settings where variables are not controlled.

So, when returning to a setting to re-do or attempt to replicate a study, we often find that the variables have changed to such an extent that replication is difficult. Furthermore, new researchers (with new subjective eyes) may catch things that the other readers overlooked.

Replication is even harder when researchers attempt to replicate a case study design in a new setting or with different participants.

Comprehension Quiz for Students

Question 1: What benefit do case studies offer when exploring the experiences of marginalized groups?

a) They provide generalizable data. b) They help express the voices of often-ignored individuals. c) They control all variables for the study. d) They always start with a clear hypothesis.

Question 2: Why might case studies be considered ideal for situations where researchers cannot control all variables?

a) They provide a structured scientific examination. b) They allow for generalizability across populations. c) They focus on one specific instance of a phenomenon. d) They allow for deep immersion in real-life settings.

Question 3: What is a primary disadvantage of case studies in terms of data applicability?

a) They always focus on the unexpected. b) They are not usually generalizable. c) They support the generation of new theories. d) They provide a holistic understanding.

Question 4: Why might case studies be considered more prone to subjectivity?

a) They always use quantitative data. b) They heavily rely on researcher interpretation, especially with qualitative data. c) They are always replicable. d) They look at a broad corpus of data.

Question 5: In what situations are experimental designs, such as those conducted in labs, most valuable?

a) When there’s a need to study rare and unusual phenomena. b) When a holistic understanding is required. c) When determining cause-and-effect relationships. d) When the study focuses on marginalized groups.

Question 6: Why is replication challenging in case study research?

a) Because they always use qualitative data. b) Because they tend to focus on a broad corpus of data. c) Due to the changing variables in complex real-world settings. d) Because they always start with a hypothesis.

Lee, B., & Saunders, M. N. K. (2017). Conducting Case Study Research for Business and Management Students. SAGE Publications.

Meir, L. (2009). Feasting on the Benefits of Case Study Research. In Mills, A. J., Wiebe, E., & Durepos, G. (Eds.). Encyclopedia of Case Study Research (Vol. 2). London: SAGE Publications.

Tetnowski, J. (2015). Qualitative case study research design.  Perspectives on fluency and fluency disorders ,  25 (1), 39-45. ( Source )

Yang, S. L. (2022). The War on Corruption in China: Local Reform and Innovation . Taylor & Francis.

Yin, R. (2003). Case Study research. Thousand Oaks, CA: Sage.

Chris

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

  • Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 5 Top Tips for Succeeding at University
  • Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 50 Durable Goods Examples
  • Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 100 Consumer Goods Examples
  • Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 30 Globalization Pros and Cons

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

  • Reference Manager
  • Simple TEXT file

People also looked at

Original research article, localizing the sustainable development goals in smart and sustainable cities: how can citizen-generated data support the local monitoring of sdgs a case study of the brussels capital region.

www.frontiersin.org

  • 1 SMIT, Studies in Media, Innovation and Technology, Faculty of Social Sciences and Solvay Business School, Vrije Universiteit Brussel, Brussels, Belgium
  • 2 Department of Applied Economics, Faculty of Social Sciences and Solvay Business School, Vrije Universiteit Brussel, Brussels, Belgium

Introduction: The Sustainable Development Goals (SDGs) serve as the global reference framework for sustainable development endeavors. However, traditional data sources, including official statistics, fall short in effectively measuring SDG performance, due to substantial gaps in the availability of reliable, timely, actionable, disaggregated, and accessible information for policy formulation. This research explores the SDG monitoring potential of citizen-generated data to enhance local environmental in the Brussels Capital Region.

Methods: Employing a qualitative approach, the study first defines and maps essential characteristics of citizen-generated data for inclusion in environmental SDG monitoring. Subsequently, expert interviews refine these characteristics and explore design requirements tailored to the Brussels Capital Region.

Results: The research culminates in a framework linking essential citizen-generated data characteristics to design requirements, ensuring data suitability for local environmental SDG monitoring.

Discussion: This framework advances the existing literature by specifically addressing local environmental SDG monitoring through citizen-generated data. It offers practical insights for local stakeholders, particularly policymakers, aiming to overcome barriers to the uptake of citizen-generated data and ultimately enhances environmental SDG monitoring in the Brussels Capital Region. The framework’s applicability in other regions or for non-environmental SDG indicators remains a potential avenue for future research.

1 Introduction

The SDGs provide a global framework with a narrative to be implemented at the local level. Despite the inclusion of input from representatives of local governments during the formulation process, the SDGs remains a global agenda with an implicit focus on the national level. Consisting of a policy framework endorsed by all 193 member states of the United Nations (UN) and are set to be achieved by 2030. Currently, there is an ongoing process of localization facilitated by subnational actors, i.e., regional, and local entities ( Greene and Meixell, 2017 ; Oosterhof, 2018 ).

The SDGs are associated with a large set of 248 indicators, which are intended to provide a comprehensive overview of progress while offering insights for future action. However, the reporting of these indicators is predominantly done at the national level, creating a challenge for subnational governments in identifying meaningful data sources to effectively measure the progress ( Greene and Meixell, 2017 ; Oosterhof, 2018 ).

There is a critical need for high-quality, accessible, timely, reliable, and disaggregated data to facilitate the measurement of progress towards the SDGs ( United Nations, 2020 ). Traditional data sources, such as official statistics, are proving inadequate for SDG measurement, necessitating innovative approaches for data collection ( United Nations, 2016 ). At the same time, a “data revolution” is unfolding, driven by technological advances that have exponentially increased the volume and variety of available data ( IEAG, 2014 ; Lämmerhirt et al., 2016 ; Fritz et al., 2019 ). The digital revolution, characterized by the widespread accessibility of broadband internet, mobile tools (e.g., dedicated apps and smart devices), and big data analytics, has transformed the landscape of information sharing, collection, and processing.

In addition, the rise of social media facilitates the promotion and encourages public participation in science projects at a range of scales, from local to global ( De Rijck et al., 2020 ; van den Homberg and Susha, 2018 ; West and Pateman, 2017 ). This phenomenon opens unprecedented opportunities for societal transformation and environmental protection ( Mahajan et al., 2022 ). Data generated directly by citizens to monitor, address, and instigate change on issues directly affecting them (referred to as citizen-generated data) holds the potential to offer a more precise and robust representation of progress. Often produced in real or near real-time, this data is firmly rooted in local contexts, amplifying citizen voices and perspectives on SDG progress, arguably including those typically marginalized and hard-to-reach populations. The production and utilization of citizen-generated data also fosters the direct, active and invested participation of individuals in advancing the SDGs ( Higgins and Cornforth, 2015 ; Hecker et al., 2019 ; Moczek et al., 2021 ). While certain initiatives - such as eBird for conservation planning, and the European bird index for biodiversity and agicultural planning - have been successful in supporting environmental action, at both EU and Member State levels, the evidence points to a gap between the policy relevance and policy uptake ( De Rijck et al., 2020 ).

In this research, we aim to offer nuanced insights to local governments regarding the barriers to the integration of citizen-generated data into monitoring initiatives like the SDGs and to suggest ways for optimization. To achieve this objective, we will construct a comprehensive framework delineating the data characteristics essential for suitable local SDG monitoring. Subsequently, we will translate these characteristics into requisite design requirements that must be considered to obtain these characteristics. The overarching goal is to formulate a set of generalized design principles that transcend diverse contexts. Citizen science projects can then refine and tailor these principles during the actual application phase.

The initial stage of this research involves desk research to identify pertinent data characteristics and their associated design requirements. Subsequently, the framework will undergo validation via a qualitative approach involving 10 semi-structured expert interviews ( Van Audenhove and Donders, 2019 ). These interviews, conducted within the domain of citizen-generated data, will seek expert perspectives on the identified characteristics and design requirements.

The article’s structure follows a logical sequence, commencing with an overview of current literature on citizen-generated data for local SDG monitoring. Subsequent sections detail the research methodologies employed and the data derived from the interviews. The article concludes with a summary and points for discussion, aiming to contribute to the discourse on citizen-generated data for local SDG monitoring.

2 Literature review

An expanding body of literature highlights the significance of localizing the SDGs. Although the SDGs were developed by and designed for national governments, the idea that there would be a crucial role for regional and local governments in the success of the goals was already present during the establishment of the Agenda 2030 ( Reddy, 2016 ). Although the Agenda 2030 explicitly recognizes the key role of cities and municipalities by dedicating a specific SDG to Sustainable Cities and Communities (SDG 11), the importance of the local level within the SDGs goes further than SDG 11. The reasoning is that efforts to achieve certain subgoals will mainly have to be done in the context of local governments (cities and municipalities), which is the closest level of governance to citizens and is a necessary level to stimulate other actors to work towards these sustainability goals ( UNDG, 2014 ). Various sources have estimated that around two-thirds or 65 per cent of the 169 SDG Targets will only be reached with a clear mandate and role for local (urban) actors in the implementation process ( Cities Alliance, 2015 ; Lafortune et al., 2019 ; OECD, 2020 ).

These targets are linked to 62 per cent of all the Official Global Indicators ( Ciambra et al., 2020 ), which brings us to the topic of local monitoring of the SDGs. To operationalise the SDGs, a form of progress measurement towards the goals will be necessary. However, the existing efforts done on the global (international) and national scale will not be sufficient at the local level because of various reasons. For example, national averages of scores on indicators can misinterpret realities on the ground and mask large regional or local disparities ( SDSN & IEEP, 2019 ).

Local SDG monitoring not only necessitates a substantial volume of data but also demands data of high quality, extensive coverage, frequent availability, and spatial disaggregation ( Fritz et al., 2019 ). Presently, SDG monitoring heavily relies on conventional and official data predominantly sourced from national statistical offices, like household surveys or administrative registers, often referred to as traditional data sources. Although essential and valuable, these traditional data sources exhibit several limitations ( Ballerini and Bergh, 2021 ). Primarily are technical constraints, involving costly and time-consuming collection processes, along with limited spatial variation and coverage which results in infrequent data collection cycles, substantial data gaps, and unrepresentative samples that systematically exclude marginalized populations, such as vulnerable minorities, and sensitive issues ( Lämmerhirt et al., 2016 ). Other often-mentioned constraints are related to the risk of manipulation by public officials to suppress contentious information, downplay their public institutions’ challenges or artificially enhance their performances, and inadequacy of traditional data sources for capturing contextual information and local knowledge ( Ballerini and Bergh, 2021 ).

Given the inherent limitations in traditional data sources, there is a growing consensus for SDG monitoring to complement them with unofficial and alternative data sources, often referred to as non-traditional data sources ( Fritz et al., 2019 ; San Llorente Capdevila et al., 2020 ; Ballerini and Bergh, 2021 ). There are various examples of non-traditional data sources. Fritz et al. (2019) divided them into 5 main categories: Earth Observations, spatial data infrastructure, official sensor networks, commercial data and citizen-generated data. This last category consitutes data voluntarily generated and gathered by individuals, herein referred to as “citizens,” with the purpose to monitor, advocate for, or instigate change in matters directly affecting them ( Lämmerhirt et al., 2016 ; Ballerini and Bergh, 2021 ). It includes a wide variety of approaches and methods, with citizens utilizing a diverse array of technologies and participatory methodologies such as community-based monitoring, crowdsourcing on online platforms, or digital sensors ( Lämmerhirt et al., 2016 ; Ballerini and Bergh, 2021 ). Typically, the production of citizen-generated data is initiated by citizens or civil society organizations and overseen by various intermediary entities, including non-governmental organizations (NGOs), academic researchers, private companies, and government agencies ( Lämmerhirt et al., 2016 ).

It is worth noting that the existing literature lacks a universally accepted definition of citizen-generated data. Depending on the context, the term is frequently used interchangeably with concepts like citizen science data, community-driven data or participatory data ( Lämmerhirt et al., 2016 ; Fritz et al., 2019 ; Ballerini and Bergh, 2021 ). Nonetheless, aligning with the perspective of Fritz et al. (2019) , we position citizen science data as an integral part of the broader concept of citizen-generated data, where individuals actively contribute data as part of a scientific research process ( McKinley et al., 2017 ; Serrano et al., 2018 ; Phillips et al., 2021 ). Presently, the prevailing focus within the literature predominantly centres on elucidating the significance of citizen science in contributing to the SDGs, as well as the utility of citizen-science data for monitoring SDG progress, as there are many examples of citizen science projects covering a diversity of domains that can contribute to the SDGs ( Fritz et al., 2019 ; Phillips et al., 2021 ). We will thus use citizen science and citizen science data as umbrella terms that cover all these terms and diverse activities.

An examination of the potential role of citizen science in defining, monitoring and implementing the SDGs was initially presented in a discussion brief published by the Stockholm Environment Institute ( West and Pateman, 2017 ; Müller et al., 2023 ) and since then the discourse on the impact of citizen science has evolved unto a growing research domain with growing output ( Müller et al., 2023 ). Current research predominantly operates on the premise that citizen science can significantly contribute to the SDGs by facilitating the envisioned environmental and social transformations outlined in the SDG agenda, driven by its democratic principles ( Sauermann et al., 2020 ; Alarcon Ferrari et al., 2021 ). These principles not only align with the SDG objective of “Leaving No One Behind” but also aim to enhance scientific productivity by involving society in the research process. Building upon these theoretical foundations, diverse stands of scientific literature have emerged. For instance, researchers have investigated how citizen science project coordinators evaluate alignment with the SDGs and the challenges encountered in contributing to them ( Moczek et al., 2021 ). Moreover, studies have documented the actual and potential scope of various forms of contribution at project, local, national, and global levels ( Fritz et al., 2019 ; Fraisl et al., 2020 ; Schleicher and Schmidt, 2020 ).

Despite acknowledging the potential of citizen science data for monitoring SDGs, the literature also highlights challenges and the limited integration of such data into monitoring and policymaking processes ( De Rijck et al., 2020 ; Alarcon Ferrari et al., 2021 ). This aligns with broader challenges like data ownership ( MacFeely, 2019 ), and the tradeoffs and negative impacts associated with big data and smart technology solutions for SDGs ( Sharifi et al., 2024 ). For instance, financial limitations in implementing these technologies in cities can hinder progress towards specific goals like poverty reduction (SDG 1), zero hunger (SDG 2), and affordable and clean energy (SDG 7). Additionally Sharifi et al. (2024) , identify trade-offs related to privacy, cybersecurity, infrastructure costs, biased decision-making that can exacerbate social inequalities (SDG 10) and limit accessibility due to the digital divide (SDG 9).

However, it is crucial to acknowledge that smart technologies also offer opportunities for achieving SDGs Sharifi et al. (2024) . Highlight successful cases where responsible data governance practices and community-driven data collection (potentially linked to citizen science) helped mitigate challenges and contributed to positive outcomes. For example, initiatives that prioritize data privacy and security while empowering communities to collect and own their data can address concerns about data ownership and promote inclusive participation (SDG 10). Therefore, a nuanced approach that carefully considers both the potential benefits and drawbacks of smart technologies is necessary to ensure they contribute to achieving SDGs in a responsible and equitable manner.

This article extends the focus to understand how citizen science initiatives can be more effectively incorporated into local environmental SDG monitoring. Therefore, we delve into the conditions for enhancing the uptake of citizen science data into local governmental SDG monitoring mechanisms.

The focus centers on SDGs with a primary environmental emphasis, including clean water and sanitation (SDG 6), sustainable cities and communities (SDG 11), responsible consumption and production (SDG 12), climate action (SDG 13), life below water (SDG 14), and life on land (SDG 15) ( UNEP, 2021 ). Citizen science activities span various environmental policy areas related to SDGs, making existing initiatives particularly relevant for providing valuable data to address these goals ( McKinley et al., 2017 ; Nascimento et al., 2018 ; Serrano et al., 2018 ; Phillips et al., 2021 ).

The exploration begins with the scholarly work of Fritz et al. (2019) , which introduces a conceptual model with five dimensions—spatial, temporal, thematic, process, and data management—each featuring distinct attributes highlighting the intrinsic value of citizen science data for SDG monitoring ( Fritz et al., 2019 ). It provides a structured way to analyze various aspects of citizen science data across key dimensions, helping to understand its strengths and limitations for specific SDG indicators.

It breaks down the evaluation of citizen science data into different dimensions, this comprehensive approach ensures a thorough assessment of diverse factors influencing data suitability for SDG monitoring. Each dimension highlights positive aspects (attributes) of citizen science data, such as denser spatial coverage, more frequent updates and diversity of domains covered. This emphasizes the potential contribution of citizen science beyond traditional data sources. The framework provides specific values or descriptions for each feature, offering a concise understanding of how citizen science data performs in each aspect. Table 1 further explains the different dimensions and attributes.

www.frontiersin.org

Table 1 . Value of citizen science data for SDG monitoring: Explanation of different features.

As the framework serves as a theoretical model, this research objective is to customize it towards policymakers and citizen science practitioners. The primary aim is to enhance its practical utility, thereby amplifying its impact and fostering broader integration of citizen science for SDG monitoring. Beyond discerning the inherent value of citizen science, we seek to illuminate the requisites for extracting actionable insights from the data. We are particularly keen on identifying potential challenges and critical considerations that could hinder realizing the full potential of citizen science for SDG monitoring.

3 Methodology

3.1 geographical focus.

The Brussels-Capital Region (BCR), situated at the heart of Belgium, serves as the focal point of our analysis. Encompassing a surface area of approximately 161.4 km 2 , 1 with a population exceeding 1.2 million inhabitants, it stands as Belgium’s sole metropolitan area and is recognized as the capital of the European Union. Administratively, the BCR comprises 19 municipalities, including the city of Brussels.

3.2 Monitoring of SDGs in the BCR

Presently, a comprehensive framework or tool specifically tailored to aid local actors in monitoring SDGs within the BCR is lacking. Some existing initiatives, such as indicators.be, 2 Donut.Brussels, 3 OECD a Territorial Approach to the SDGs, 4 and SDSN European Cities SDG Index, 5 offer data and indicators for the region as whole. However, these initiatives predominantly operate at the regional level, overlooking the unique composition of the 19 individual municipalities within the BCR.

At the local level, some municipalities in the BCR collect data related to themes within the environmental SDGs as part of own developed climate plans in response to a project call initiated by the regional government. These efforts mainly rely on traditional data sources, with the potential of citizen science data sources largely remaining untapped.

3.3 SDG in ACTION

The growing demand for support for local implementation of the SDGs resulted in the SDG in ACTION PhD project. 6 This applied PhD, funded by the Brussels Capital Region (Innoviris) and implemented by Studies in Media and Innovation (SMIT) and IDEA Consult. The primary objective is to provide support to local public and private stakeholders within the BCR in their efforts to monitor progress towards local SDG targets. This study is done in the context of this research project.

3.4 Citizen science projects in the BCR

The BCR today hosts several noteworthy citizen science data projects, particularly in the domain of Air Quality monitoring, including projects such as CureuzenAir BXL, compAIR, AIRCasting, expAIR. Despite the existence of these projects, there is currently no comprehensive public portal that consolidates information on past, ongoing or upcoming citizen science projects within the region.

However, there are available resources that offer partial insights into the landscape of citizen science in the BCR. The EU-citizen.science website 7 features some projects associated with the BCR, but the coverage is somewhat limited. Furthermore, the website from the Vrije Universiteit Brussel 8 provides an overview of some of the projects researchers from the university are involved with.

3.5 Data collection method

Our data collection adopts a qualitative methodology, employing semi-structured interviews with data experts in citizen science data. This method aims to extract bottom-up insights into various aspects related to the data generated during citizen science data projects.

In the initial phase, a comprehensive literature review and desk research were conducted to orient the research, identify relevant experts, and construct a topic list to guidance the interviews ( Van Audenhove and Donders, 2019 ).

The interviews were conducted using a semi-structured topic list, providing flexibility to guide the conversation and adapt questions based on the interviewee’s responses. Open-ended questions were employed to allow interviewees to provide detailed and nuanced responses, thereby enabling a deeper understanding of their perspectives and often yielding valuable additional information. The interviews were structured into five broad, predefined sections:

1. Welcome and introduction: Communicating the purpose of the interview, its relation to the SDG in ACTION project, and outlining the interview process.

2. Introductory questions and general knowledge: Exploring the interviewees’ familiarity with the SDG framework and non-traditional data sources.

3. Citizen-generated data sources and the SDGs: Delving further into different elements of the framework making abstraction of the local context in the BCR.

4. Citizen-generated data sources for the BCR: Addressing questions on the concrete applicability of citizen-generated data for the BCR and ideas for future projects.

5. Closing: Conduct a short debrief, providing information on the next steps in the data analysis process.

In the subsequent phase, participants were selected based on specific characteristics relevant to the research question. The selection involved reaching out to the Flemish Knowledge Centre for Citizen Science (Scivil) 9 to support the identification of suitable data experts. We employed a purposive sampling strategy, a non-probability method that involves selecting participants based on specific characteristics relevant to the research question. This strategy aimed to include a diverse range of experts and projects, obtaining varied perspectives and activities without seeking population generalization ( Bryman, 2012 ).

Invitations were extended to experts with proficiency in citizen science data related to diverse environmental topics, including air and water quality monitoring, biodiversity, and with knowledge on a variety of data gathering techniques. The interviewed experts possess various backgrounds, representing both governmental and non-governmental stakeholders. Further details about the background of the participants and their expertise are presented in Table 2 below.

www.frontiersin.org

Table 2 . Background of participants and expertise.

The expert interviews, totaling 10, were conducted through video conferencing on Teams, each lasting approximately 1.5 h. Given the multilingual nature of the BC, a multilingual approach was adopted, conducting interviews in English, French or Dutch, based on the participants’ preferences. The interviews were systematically recorded, automatically transcribed, and supplemented with notes for subsequent data analysis where thematic analysis was employed to identify and analyze themes within the data ( Herzog et al., 2019 ).

This study adopts the framework outlined by ( Fritz et al., 2019 ) to structure the insights gathered from our expert interviews. However for improved clarity, we introduce two overarching layers that consolidate the original set of five dimensions. The Data scope layer encompasses the spatial, temporal, and thematic aspects of citizen science data. It aligns with the prevalent understanding of data scope as the breadth and depth of information captured within a dataset ( Abraham et al., 2019 ). Conversely, data governance focusses on the management, processing, and overall governance structure of the citizen-science projects. It incorporates the formalization of data policies, standards, and procedures ( Abraham et al., 2019 ), encompassing the data management and process dimensions from the original framework. This way it emphasizes the interconnected nature of data collection, processing and sharing. Additionally, we integrate data quality into the framework as a separate dimension under data governance, recognizing its importance as a barrier to citizen-generated data utilization in monitoring efforts ( Fritz et al., 2019 ). This dual-layered approach provides a comprehensive structure for organizing and analyzing expert insights. It enhances the clarity and coherence of our analysis, facilitating a holistic understanding of the various facets associated with citizen science data within data scope and data governance. Table 3 below outlines the key elements of the adapted framework, including layers, dimensions, and attributes.

www.frontiersin.org

Table 3 . Adapted framework with overarching layers.

4.1 Data scope

This layer plays a crucial role within the overall framework and delves deeper into three key dimensions that define the scope and suitability of citizen science data for monitoring SDGs: 1) Data extending traditional data boundaries, 2) fit for purpose, and 3) data representativeness.

4.1.1 Data boundary extension

Our experts underscore the constraints of traditional data sources regarding the coverage, resolution and extent these sources offer for monitoring within their domains of expertise like biodiversity, air- and water quality. Extending beyond these conventional boundaries, citizen science data has the potential to transcend the limitations inherent in traditional data sources. This can take various forms, e.g., spatial, temporal, or thematic, leading to attaining a level of granularity that aligns with the specific information requirements of policy formulation. Moreover, using citizens’ capacity to articulate their sentiments and preferences on various matters can also be a way they go beyond the boundaries of traditional data. This approach facilitates a nuanced understanding of issues like pollution, surpassing the confines of (annual) emission measurements by incorporating citizens’ perceptions and experiences, for example, including the assessment of citizens’ perceptions of odors.

Citizen involvement enables a cost-effective expansion of measurement points, supplementing traditional data sources and ensuring geographical coverage in areas that may otherwise be overlooked by the traditional governmental measurement stations like remote, hard-to-reach, or inaccessible locations. The “CurieuzenAir BXL” project in the BCR, serves as a good example, as it employed citizen science sensors for air quality monitoring (concentration of NO2) in the region to provide a more intricate perspective on air quality. By involving citizens asking them to attach a sensor to their houses it covered Air Quality at street level.

Telraam serves as another illustration, employing sensors affixed to citizens’ windows to provide both citizens and local administration with a comprehensive portrayal of neighborhood traffic. The insights derived from such initiatives surpass the granularity achieved through a limited number of counting loops or cameras installed by a local government.

4.1.2 Fit for purpose

While the expansion of existing traditional data boundaries is a prerequisite for citizen science data to become interesting for governments to use the data in monitoring efforts, our experts emphasize the need for a careful balance to ensure that the collected data aligns proportionally with these monitoring objectives. The gathered data should be proportional and adequate for the intended monitoring purposes, which could be in conflict with typical citizen science approaches where the purpose is defined before data collection begins. In conventional citizen science projects, clear goals and objectives are established, and data is collected specifically to address those questions. This approach ensures that the data is relevant, efficient, and manageable for analysis and decision-making in the context of the project.

Nevertheless, there are instances where citizen science projects may adopt a more open-ended exploration approach, allowing participants to generate valuable data without predefined objectives. While this may initially seem divergent from the traditional method, such exploratory projects can yield data that, under retrospective assessment, proves valuable for specific monitoring efforts, particularly at the local level.

This highlights challenges arising from the abundance of citizen science data for local governments. While such data offers enhanced granularity, it simultaneously presents hurdles—particularly in terms of increased data processing demands. Local governments, constrained by human and technical capacity and expertise, may find managing this influx challenging. Additionally, for monitoring purposes, data should offer information that is easily interpretable, necessitating only the granularity essential for its intended purpose and thus appropriate scales.

The pursuit of super-accurate data may not always be necessary ( Bowser et al., 2020 ). In certain scenarios, a reduced level of accuracy may still be sufficient for monitoring purposes. An often-cited example by our experts is related to the temporal scale. For monitoring purposes, real-time data is often not an absolute requirement. Rather, the data should be updated at a frequency corresponding with the analysis or monitoring needs of the government.

Furthermore, a call is made for broader Application Programming Interface (API) capabilities for specific policy applications and operational decision-making. APIs function as intermediaries that enable seamless communication between distinct software programs, thereby facilitating real-time data visualization on dashboards and remote adjustments to sensor settings. An illustrative example involves the case of Telraam, where adjusting the API’s data measurement frequency enabled a more accurate assessment, showcasing the need for tailored API solutions to optimize data usability for policy applications. In the context of a school street, 10 operational from 7:30 to 8:30 am and 3:30 to 4:30 pm, the sensors initially recorded data on an hourly basis, spanning from 7:00 to 8:00, 8:00 to 9:00 am, and so forth. Consequently, the data encompassed half-hour intervals, during which the school street was both opened and closed. To address this limitation, a modification to the API was implemented, shifting the data measurement frequency to a quarter-hour basis. This adjustment proved instrumental in comprehensively capturing and mapping the impact of the school street on the surrounding environment.

To allow for adequacy in terms of available data, citizen science projects need to profoundly understand the data needs of the local government for monitoring purposes. In the context of exploring viable data for SDG monitoring, this necessitates a thorough examination of the SDG framework itself. While ensuring thematic alignment with the SDGs is crucial, our experts express the challenge regarding the limited knowledge among citizen science projects of the SDG framework and the difficulty of understanding its intricate nature.

Concerning the first element, our experts highlight a limited integration of the SDGs within citizen science data projects. Instances where the SDGs are incorporated tend to be associated with mandatory links to proposals for external funding, indicating a somewhat instrumental use rather than intrinsic alignment. Addressing the second element, our expert’s express challenges arising from the complex nature of the SDGs. One expert mentioning the ‘academic nature’ of the SDGs as a barrier, presenting a difficulty in comprehending the framework and identifying relevant indicators. Another expert, focusing on citizen science data related to biodiversity, exemplifies the struggle in monitoring specific SDG targets (15.2 and 15.9) without having a comprehensive understanding of the broader SDG framework. These examples underscore the common difficulties faced by projects, emphasizing the need for a more accessible interpretation of the SDGs. Existing knowledge often centers around the 17 goals themselves, with limited awareness of the underlying framework of subgoals and indicators. There is a call for translating the SDGs into a format adapted to the local context of the BCR, ensuring a clearer and more relevant perspective aligned with local needs.

4.1.3 Data representativeness

A recurrent theme highlighted by our experts revolves around the vital consideration of the representativeness of the data being collected by citizen science data projects. Inclusivity emerges as central, emphasizing the need to avoid data biases and ensure representation in line with the fundamental principle within the SDG framework of “Leave No One Behind” (LNOB) ( Lämmerhirt et al., 2016 ). The literature underscores citizen science data as promising in this regard, enabling the capture of bottom-up insights on under-reported issues, facilitating the expression of counter-narratives of sustainable development, and the genuine empowerment of all voices ( Lämmerhirt et al., 2016 ).

While citizen science data offers promise in terms of inclusivity, current trends often involve participation biased towards individuals with higher socio-economic status. As articulated by one expert, “challenges arise in achieving diversity, with prevailing trends often featuring the participation of economically privileged individuals, typically of white ethnicity, male, possessing advanced degrees, and of an older age.” Consequently, projects must broaden their participation pool beyond those who already possess resources like time and capital to engage in citizen science activities.

It is important to recognize that citizens engage in citizen science activities for various reasons, ranging from personal interest in the topic, financial incentives for data acquisition, a desire for knowledge enrichment throughout the project (such as understanding the functioning of sensors or the overarching topic), a sense of community belonging, or a genuine intention to contribute positively. Acknowledging and catering to these diverse motivations is crucial for attracting a wide range of participants.

Successful citizen science projects should actively involve underrepresented groups, fulfilling the overarching purpose of democratizing knowledge for policy formulation and implementation. To address this challenge, deliberate efforts are needed to include these underrepresented groups and amplify unheard voices.

The importance of representativeness varies depending on the type of data being collected, as the imperative for representativeness varies across domains. In biodiversity topics, such as counting vulnerable species, the characteristics of the citizen collecting the data may have less impact. However, in areas like environmental issues linked to socioeconomic factors, representativeness becomes critical. This is exemplified by the results of CurieuzenAir BXL project, were 3,000 participants across diverse socioeconomic areas within the territory of the BCR used citizen science sensors to map air quality, revealing correlations between air quality and the socio-economic status of specific neighborhoods.

Recognizing that not all projects may have the means to collect data on such a broad scale, our experts suggested measures to enhance the representativeness of the data sample, by collaborating with intermediaries. One suggested approach is to engage with social organizations, directly involving them to facilitate the deployment of citizen-science sensors. However, it is essential to acknowledge the financial constraints often faced by these organizations. Another viable method involves building partnerships with organizations to access their knowledge regarding underrepresented groups. For example, schools can be valuable contributors, leveraging their existing data to delineate the proportion of certain demographics within a population, thereby augmenting representativeness. In the context of the BCR, Flemish schools participating in initiatives like the Flemish GOK-Indicators 11 related to equal educational opportunities, can provide data on parameters related to individuals in need of support.

Privacy concerns, in compliance with regulations like the GDPR, demand careful attention. Issues may arise in accessing vulnerable groups, such as the Roma community, highlighting the ethical and practical challenges of ensuring inclusivity in citizen science initiatives.

4.2 Data governance

Within the overarching framework, data governance plays a critical role. It is important to understand its two key components, data process and data management.

4.2.1 Data process

4.2.1.1 citizen science for the long haul.

For data to be truly valuable, it must extend beyond the temporal boundaries of individual projects. Our experts highlighted the pronounced challenge of maintaining projects and ensuring continuity in data gathering. Particularly given the project-based nature of many citizen science initiatives, it requires attention to the intermittent nature of data collection inherent in such projects. Seamless integration of these projects into governmental monitoring initiatives necessitates regular campaigns or continuous data collection.

This often originates from limited funding, restricting projects to data collection only over a limited period. The financial aspect is a significant consideration in the realm of citizen science projects. For instance, while citizen science sensors may offer cost-effective solutions in specific domains, it is imperative to acknowledge that maintaining projects over an extended period can still incur substantial expenses. A case in point is the CurieuzenAir BXL project, where the tubes used to measure air quality incur regular laboratory costs of around 10€. Although manageable over a short period, these costs accumulate over time. Thus, to enhance the sustainability of citizen science initiatives, a comprehensive understanding of the potential long-term expenses is crucial. These long-term costs are related to project maintenance, including for example, data collection, data management, sensor calibration, data storage and hosting, 12 and ongoing support. A concrete illustration of such costs is evident in the case of one of our experts, who incurs a monthly expenditure of € 1,000 for data hosting through Amazon Cloud Services.

Conversely, certain factors contribute to cost reduction, such as leveraging voluntary technicians for the maintenance of measurement instruments like sensors. For instance, citizens actively engage in cleaning water quality sensors and conduct checks by capturing and sharing pictures with a centralized database. If organizers identify trends in the sensor data that appear inaccurate, they can remotely validate the sensor quality. This collaborative effort not only reduces costs but also enhances the efficiency of data maintenance.

Furthermore, the maturity level of technology emerges as a significant factor influencing costs. Advancements in technology can lead to more cost-effective solutions, streamlining the overall expenses associated with citizen science initiatives. An expert specialized in citizen science water sensors elucidated that the maturity level of sensor technologies varies across different fields. Notably, air quality sensors have advanced significantly, providing cost-effective options that facilitate straightforward installations by citizens, thus being considered “plug and play.” This high level of maturity in the technology translates into lower usage costs. In contrast, water quality sensors may lag in maturity with the development and accessibility of affordable and user-friendly water quality sensors still in nascent stages, making them less seamlessly integrable or “plug and play” compared to their air quality counterparts, potentially contributing to higher costs.

Some initiatives have transitioned into long-term data providers, establishing a business model to operate as sustainable service providers for local governments. Telraam, active in the BCR, is a prime example. Originating as a project that developed a traffic-counting sensor, it transformed into an enterprise with its own business model, serving as a sustainable data provider to local governments. The success of such initiatives relies on demonstrating the added value of their data to the local government, complementing existing traditional data sources fulfilling specific data needs, and attaining the requisite volume to maintain cost-effectiveness.

Identifying and addressing data blind spots regarding traffic counting, particularly in less-travelled secondary roads, holds significance for local administrations, prompting a willingness to invest in such data. Traffic count data proves valuable not only for its direct application, but also as proxy data for urban livability, encompassing factors like traffic density and share per type of road users, or air quality assessment, particularly in understanding emissions related to motorized traffic.

Notably, the design of the sensor itself contributes significantly to the success of initiatives like Telraam. Tailoring sensors for user-friendliness and application specificity, enhances citizen engagement, as evidenced by Telraam’s approach. This underscores the importance of citizens recognizing the value of the data they collect, thereby increasing their likelihood of remaining long-term engaged data providers. Again, Telraam serves as an exemplary model in this context. For citizens, the data holds significance as it enables them to objectively address concerns about local traffic conditions in their streets, fostering a more effective dialogue with their local government. This not only contributes to citizen empowerment but also cultivates a heightened sense of ownership and involvement in community affairs.

Intrinsic motivation also emerges as a vital factor for the sustainability of citizen science projects, where managing engagement fatigue poses a specific challenge. Sustaining motivation for data sharing and continued participation beyond the project lifecycle requires careful expectation management and clear communication about long-term benefits. Strategies to actively engage and motivate participants in the data collection process are imperative. While organizers may influence certain elements, such as sensor design and engagement strategies, external factors like citizens moving to different locations may remain beyond their control. Open communication channels and a transparent process help build trust and encourage active participation. This also is demonstrated in the need for legal and ethical considerations as discussed below.

4.2.1.2 Ethical and legal considerations

Our experts underscore the significance of ethical and legal considerations, particularly in interactions with participating citizens during projects. Citizens should be well-informed about the handling of their data: Transparency and clarity regarding the use and transfer of data are crucial to establishing trust. Consequently, clear agreements, such as an informed consent form, should be in place, outlining the terms and conditions governing the project’s objectives, the purpose of data collection, data usage, ownership and potential data transfer to (governmental) data repositories.

4.2.2 Data management

4.2.2.1 fair principles in practice.

In the sphere of citizen science initiatives, effective data management plays a pivotal role, encompassing a variety of tools dedicated to the facets of data collection, storage, and dissemination. Its significance lies in facilitating collaboration between citizens and institutions throughout the entire project lifecycle. Incorporating FAIR principles—ensuring data is Findable, Accessible, Interoperable, and Reusable—form a strong foundation for data management. It ensures high quality data that can be effectively utilized for scientific research, policymaking, and community engagement ( Wilkinson et al., 2016 ; Bowser et al., 2020 ; San Llorente Capdevila et al., 2020 ; Fritzenkötter et al., 2022 ).

Exemplary projects, such as WorldFAIR in the realm of Biodiversity, are cited by our experts as model for implementing FAIR principles and advancing data management practices in citizen science initiatives. These endeavors contribute to the creation of a robust and accessible data ecosystem beneficial to both the scientific community and the broader public. However, despite the significant improvements FAIR principles offer, specific concerns remain, including data quality, software compatibility, and content related trustworthiness ( Koedel et al., 2022 ). Addressing these challenges necessitates additional approaches for ensuring effective data linkages and joint interpretation. While our experts recognize these challenges, they also offer valuable insights into key elements supporting the implementation of FAIR principles.

Notably the publication of data was highlighted, emphasizing the importance of publishing citizen science data in a professional manner, in adherence to appropriate metadata standards. While certain fields lack metadata standards, the development of such standards becomes essential to establish a structured framework for describing data and its origin. This contributes to enhanced data interoperability, facilitating seamless collaboration and analysis across diverse projects and domains. For instance, the use of the INSPIRE Metadata Regulation, a common metadata standard for the Infrastructure for Spatial Information in the European Community (INSPIRE), 13 ensures transparency in data provenance - clarifying the origin, creation, and propagation of the data ( Imran and Agrawal, 2017 ). Standardizing data in a uniform and interoperable manner is also exemplified by the Darwincore in Biodiversity data which includes a glossary of terms intended to facilitate the sharing of information about biological diversity by providing identifiers, labels, and definitions. 14

Our experts furthermore underscored the importance of making data accessible in an open data format—preferably a machine-readable format - as still in many cases data is made available in human-readable format (excel, pdf) (e.g., CSV, JSON, XML) ( EC JRC, 2020 ). Particularly noteworthy is the progress made in the biodiversity data domain through the implementation Linked Open Data (LOD). Notably, certain other fields have already embraced LOD practices, showcasing it as a promising model for adoption in diverse citizen science initiatives. This trend suggests that leveraging linked open data principles can contribute significantly to enhancing the accessibility and interoperability of data in various domains within the realm of citizen science.

The integration of data from citizens-science data projects and applications into monitoring systems presents challenges, particularly in aligning citizen science projects with existing proprietary applications of local governments. This is related to the data structure, but also to the way the data is introduced into these governmental systems. While the proposition of constructing an appropriate API for integrating data into monitoring systems is widely acknowledged ( EC JRC, 2020 ), one of our experts mentions that challenges arise due to several aspects, like the ownership of data by citizens, or when the data enters the governmental system, the control is lost over the data from the report or the feedback back to the citizens which is supposed to be given.

4.2.2.2 Data quality assurance

The recurring theme of concern surrounding data quality is underscored by our experts, who recognize factors contributing to lower data quality, such as sensor quality and external influences like power cuts. Proposed solutions include efforts to improve the quality of low-quality sensors and employ techniques like time series analysis to address outliers. Additionally, the use of expert calibration algorithms has been suggested.

To preserve data quality in certain fields, a multi-layered approach to data sourcing is showcased, involving the integration of data sets from distinct sources, with the inclusion of citizen science sensor data. In the context of air quality measurements, three layers are delineated. The first layer consists of reference stations adhering to European norms, thereby upholding high standards of testing and calibration, ensuring reliability and accuracy. The second layer encompasses commercial sensors, which, despite their cost-effectiveness, adhere to rigorous testing and calibration criteria. The third layer are citizen science sensors, characterized by lower components and minimal calibration, manifesting limitations in terms of sophistication and calibration precision.

As Table 4 below illustrates, each layer has its strengths and weaknesses. Reference stations adhere to the standard but lack widespread coverage. Commercial sensors offer good value but might miss local details. Citizen science sensors, while valuable for broader understanding, may require further refinement for accuracy. Notwithstanding potential limitations inherent in each layer, the integration of data from all three layers is crucial for a comprehensive and nuanced understanding of air quality, from large scale trade trends to localized hotspots.

www.frontiersin.org

Table 4 . A comparison of three layers of Air Quality Sensors.

Given the prevalence of low-cost sensors in citizen science, where calibration is often a one-time procedure at the factory, and discrepancies among sensors may persist even after calibration, a recommended practice is subjecting low-cost sensors to a sensor comparison experiment for validation prior to distribution to citizen.

The integration of APIs can significantly enhance the quality of citizen science data. APIs enable direct data storage within databases upon collection, streamlining the quality assurance and quality control (QA/QC) processes. They contribute to improved data integrity by identifying outliers, detecting drift, and ensure consistency with neighboring data points. Established data quality measures traditionally applied to official data, such as positional and thematic accuracy, temporal currency, completeness and representativeness across spatial and temporal dimensions, and appropriateness for the intended purpose ( Fritz et al., 2019 ), can seamlessly be implemented within citizen science projects through the use of APIs.

Apart from actual data quality, a significant aspect revolves around potential misperceptions concerning data quality, recognizing that citizen science data may not always universally be accepted as qualitatively sound. Such considerations underscore the importance of end-users of the data, like civil servants, policy- and decision-makers, possessing expertise in data interpretation. It is emphasized in our interviews that a singular anomalous data point does not necessarily discredit the entirety of the dataset, aligning with the concept of fitness for purpose and emphasizing the contextual significance of data quality and its application ( Bowser et al., 2020 ). Furthermore, the depth and quality of the data processing also play a crucial role in shaping the final dataset and thus the information presented to the end-users.

The complexity of data analysis is linked to an understanding of legal thresholds, as exemplified in the context of air quality data. Recognition is essential that the presence of an average threshold does not necessarily deem the data unreliable if occasionally surpassed. In the case of air quality assessments, where various temporal thresholds are in place, a singular instance of surpassing the threshold does not inherently compromise the overall quality of the measurement data.

Expertise regarding data interpretation within local governments is paramount to ensure that collected data is effectively analyzed and interpreted to derive meaningful insights. Support from higher governments can play a vital role in providing the necessary resources, guidance, and validation for the data interpretation process. However, treating local governments as a homogeneous group does not correspond with reality, as various sizes, organizational capabilities, and data needs were highlighted during our interviews. For instance, the BCR encompasses 19 municipalities, with the city of Brussels standing out due to its significant population and hosting numerous EU-related and international institutions, in contrast to smaller municipalities within the BCR. A higher government within the BCR can play a pivotal role, not only in providing expertise regarding data interpretation but also regarding infrastructure for data sharing and analysis.

4.3 Refined framework

Building upon the foundational work of Fritz et al. (2019) , this revised framework articulates essential data characteristics and design requirements for effective citizen science data integration into local environmental SDG monitoring. The framework seeks to provide a nuanced understanding of the intricacies involved, fostering collaboration between local governments and citizen science projects.

Under the data scope layer, crucial characteristics are identified to render citizen science data valuable for local SDG monitoring initiatives. These encompass:

1. Data extending traditional boundaries: Citizen science data should surpass limitations of traditional data sources, expanding spatial, temporal, and thematic coverage, including the incorporation of citizens’ perceptions and experiences.

2. Fit for purpose: The data should be aligned with monitoring objectives, finding a balance between the right granularity and manageable data processing.

3. Representativeness: Emphasizes the importance of inclusivity to avoid biases in citizen science data.

Under the data governance layer, dimensions are perceived as constituting design requirements, outlining necessary processes and practices for well-designed citizen-generated data projects. These encompass:

1. Sustainability: Highlights the need to maintain data gathering beyond individual citizen science projects for continuous data collection.

2. Ethical and legal considerations: Emphasizes the importance of ethical and legal considerations in citizen science projects, encompassing transparency, informed consent, and considerations of data ownership.

3. Effective data management: Explores the practical implementation of FAIR principles (Findable, Accessible, Interoperable, Reusable) in citizen science data management. Emphasis is placed on professional data publication, adherence to metadata standards, and the adoption of open data formats.

4. Data quality assurance: Recognizes the importance of data quality in citizen science projects, addressing challenges including sensor quality, external influences, and the importance of the right data interpretation.

The revised framework furnishes a more intricate and nuanced comprehension of the various aspects influencing the value of citizen-generated data for governmental monitoring. It addresses challenges, provides examples, and offers practical considerations across diverse aspects, positioning it as a valuable tool for comprehending and executing citizen science initiatives in the context of governmental monitoring. Table 5 below presents an overview of the revised framework.

www.frontiersin.org

Table 5 . Overview of the revised framework with data characteristics and design requirements.

5 Limitations

The research methodology employed in this article presents certain constraints that warrant consideration. The geographic focus is primarily on the BCR, and caution should be exercised when generalizing the findings to other regions. The intentional choice to closely examine an individual locality underscores the research emphasis on advocating for a bottom-up refinement of SDGs, aligning with existing research recommendations.

While the presented methodology offers a valuable framework, its application in diverse contexts necessitates careful consideration of site-specific factors such as the number of relevant citizen science initiatives providing data and the degree of local administrative support. These factors significantly influence its potential contribution to SDG monitoring.

A limitation arises from the sample set used for interviews, which primarily comprised of data experts and which was limited to 10 interviews. This selection choice imposes certain restrictions on the breadth of perspectives considered.

Furthermore, while the study engaged with data experts and subject specialists, it did not involve stakeholders from the local governments themselves, especially at the political level. Future research endeavors may benefit from exploring how insights from this perspective align with of diverge from those gathered, thereby contributing to a more comprehensive understanding. It is acknowledged that local (political) preferences regarding data-driven governance exist, and their inclusion in future investigations would offer valuable insights into these nuanced dynamics.

6 Conclusion—implications—applications

This article emphasizes the growing recognition of the crucial role of local governments in achieving the SDGs and thus the need for monitoring them at the local level. Traditional data sources, while essential, have limitations, leading to a call for supplementing them with non-traditional sources, like citizen science data. Existing literature explores the significance of citizen science data in contributing to and monitoring SDG progress, acknowledging challenges and limited integration into policymaking. It extends this discussion by focusing on how citizen science data can be effectively incorporated into local environmental SDG monitoring, investigating the barriers for uptake of citizen science data into local SDG monitoring efforts, with a focus on environmentally relevant SDGs. The exploration builds upon a conceptual model introduced by Fritz et al. (2019) , which outlines dimensions highlighting the intrinsic value of citizen science data for SDG monitoring.

The study focusses on the BCR and involves qualitative data collection through 10 in-depth semi-structured interviews with citizen science data experts from various backgrounds. We started restructuring the original five dimensions of the conceptual model into two overarching categories—data scope and organizational data governance—which provides a clearer understanding of the complexities involved. Our examination of the conceptual framework unveils profound insights into the dimensions of data scope and data governance.

In the data scope layer, three key dimensions are highlighted. The first one is the extension of the boundaries of traditional data, surpassing them in terms of coverage, resolution, and granularity. It offers a nuanced understanding of issues such as air pollution and traffic flows. However, while expanding these data boundaries is essential, there is a need to ensure data aligns with monitoring objectives. Which brings us to our second dimension, the need for the data to be fit for purpose. Challenges include increased data processing demands and the importance of understanding local government’s data needs. Alignment with the SDGs is crucial, but projects often lack awareness and struggle with the complex nature of the SDG Framework. A third dimension revolves around the representativeness of the data being collected. Inclusivity is crucial to avoid biases and align with the SDG principle of “Leave No One Behind.” Challenges arise in achieving diversity, with current trends involving economically privileged individuals. The integration of underrepresented groups significantly enhances the richness of citizen science data, contributing to the democratization of knowledge crucial for informed policy formulation. Collaborations with intermediaries and educational institutions are suggested to enhance representativeness.

Regarding the data governance layer, there are two main components: the data process and data management. Within the data process, our analysis highlights the challenges of sustaining citizen science data projects beyond one-off initiatives. Often financial constraints lead to the temporal nature of data collection, presenting hurdles that require careful planning and resource allocation. Although there are examples of projects that successfully transitioned into sustainable service providers for local governments. The importance of citizen engagement is underscored, acknowledging the challenge of managing engagement fatigue. Furthermore, ethical and legal considerations are highlighted as crucial, emphasizing transparency, informed consent and clear guidelines for data ownership in citizen science data projects.

In the realm of data management, adherence to FAIR principles is identified as key to facilitating collaboration and ensuring the quality and usability of generated data. However, challenges persist in standardizing data practices, publication in open formats, achieving interoperability, and data integration with other existing data sources. Regarding data quality, our experts emphasize that not only the actual quality of citizen science data is a recurring concern, but they underscore the need for expertise in data interpretation, with higher governments in a supporting role.

Based on qualitative research, we enhanced the conceptual model for the BCR with data scope characteristics, including data extending traditional boundaries, the fitness for a specific objective and ensuring the representativeness of the collected data. The necessary design requirements are situated within the domain of data governance, including considerations related to the sustainability of data collection processes, ethical and legal dimensions, standardized data publication, data accessibility, data integration and data quality assurance.

Our results provide local stakeholders and in particular policymakers practical insights into how the barriers for uptake of citizen-generated data into local SDG monitoring of environmental indicators can be overcome. As such, we want to improve the uptake of citizen-generated data projects into Environmental SDG monitoring in the BCR.

The findings underscore the multifaceted nature of integrating citizen science data into policy, highlighting the need for a holistic and adaptive approach. Here, “holistic” refers to the inclusive consideration of diverse dimensions within the framework, while “adaptive” signifies the recognition that contextual factors play a crucial role, thereby acknowledging the limited generalizability. Effectively integrating citizen science data into SDG monitoring at the local level necessitates addressing challenges related to data scope, data governance and context-specific considerations.

In future research endeavors, this framework can be employed to substantiate the practical impact and iteratively refine the underlying framework. Additionally, its applicability may extend to other geographical territories or be adapted for application to citizen-generated data sources directed towards non environmental SDG indicators.

Data availability statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Author contributions

KB: Conceptualization, Data curation, Formal Analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing–original draft, Writing–review and editing. LV: Methodology, Supervision, Writing–review and editing. CV: Supervision, Writing–review and editing. LT: Supervision, Writing–review and editing. RH: Supervision, Writing–review and editing.

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. The research was done in the context of the PhD project titled “SDG in ACTION—Data Driven Coaching of Local Communities in Achieving SDG Goals” which is subsidized by the Brussels Capital Region—Innoviris.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

1 https://bisa.brussels/wist-je-dat/162-4-km2-is-de-oppervlakte-van-het-brussels-hoofdstedelijk-gewest

2 https://indicators.be/en/t/SDG/

3 https://donut.brussels/en/brussels/en-macro-portrait/

4 https://www.oecd-local-sdgs.org/index.html

5 https://euro-cities.sdgindex.org/#/

6 https://smit.vub.ac.be/phd-project-sdg-in-action-data-driven-coaching-of-local-communities-in-achieving-sdg-goals

7 https://eu-citizen.science/projects?keywords=&orderby=&country=BE&status=&hasTag=&difficultyLevel=&topic=&participationTask=

8 https://www.vub.be/en/our-research/our-vision-and-mission/participatory-and-community-based-research/citizen-science

9 https://www.scivil.be

10 A public road in the vicinity of an educational establishment where motor vehicles are temporarily barred at the entrances during certain hours.

11 Gelijke OnderwijsKansen or Equal Education Opportunities. Indicators on language spoken at home, school allowance, educational level parents.

12 Storing the data on a stable and accessible web platform ( https://www.gbif.org/data-hosting ); costs can incur costs for a subscription on a cloud-based platform.

13 https://rdamsc.bath.ac.uk/msc/m66

14 https://dwc.tdwg.org

Abraham, R., Schneider, J., and Vom Brocke, J. (2019). Data governance: a conceptual framework, structured review, and research agenda. Int. J. Inf. Manag. 49, 424–438. doi:10.1016/j.ijinfomgt.2019.07.008

CrossRef Full Text | Google Scholar

Alarcon Ferrari, C., Jönsson, M., Gebreyohannis Gebrehiwot, S., Chiwona-Karltun, L., Mark-Herbert, C., Manuschevich, D., et al. (2021). Citizen science as democratic innovation that renews environmental monitoring and assessment for the sustainable development goals in rural areas. Sustainability 13 (5), 2762. doi:10.3390/su13052762

Ballerini, L., and Bergh, S. I. (2021). Using citizen science data to monitor the Sustainable Development Goals: a bottom-up analysis. Sustain. Sci. 16 (6), 1945–1962. doi:10.1007/s11625-021-01001-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Bowser, A., Cooper, C., De Sherbinin, A., Wiggins, A., Brenton, P., Chuang, T.-R., et al. (2020). Still in need of norms: the state of the data in citizen science. Citiz. Sci. Theory Pract. 5 (1), 18. doi:10.5334/cstp.303

Bryman, A. (2012). Social research methods . Fourth edition. Oxford University Press .

Google Scholar

Ciambra, A., Fernández, A., Calvete, A., Miraglia, M., and Herrera-Favela, L. (2020). Guidelines for voluntary local reviews: Volume one—a comparative analysis of existing VLRs (volume 1; guidelines for voluntary local reviews). United cities & local governments (UCLG) and UN-habitat. https://unhabitat.org/sites/default/files/2020/07/uclg_vlrlab_guidelines_july_2020_final.pdf .

Cities Alliance (2015). Sustainable Development Goals and Habitat III: Opportunities for a Successful New Urban Agenda [Discussion Paper]. Cities Alliance. Available at: https://www.citiesalliance.org/sites/default/files/Opportunities%20for%20the%20New%20Urban%20Agenda.pdf .

De Rijck, K., Schade, S., Rubio, J.-M., and Van Meerloo, M. (2020). Best practices in citizen science for environmental monitoring: commission staff working document. [75pp.]. doi:10.25607/OBP-1779

EC JRC (2020). Scaling up citizen science: what are the factors associated with increased reach and how to lever them to achieve impact . Luxembourg: Publications Office . doi:10.2760/00926

Fraisl, D., Campbell, J., See, L., Wehn, U., Wardlaw, J., Gold, M., et al. (2020). Mapping citizen science contributions to the UN sustainable development goals. Sustain. Sci. 15 (6), 1735–1751. doi:10.1007/s11625-020-00833-7

Fritz, S., See, L., Carlson, T., Haklay, M., Oliver, J. L., Fraisl, D., et al. (2019). Citizen science and the united Nations sustainable development goals. Nat. Sustain. 2 (10), 922–930. doi:10.1038/s41893-019-0390-3

Fritzenkötter, J., Hohoff, L., Pierri, P., Verhulst, S., Young, A., and Zacharzewski, A. (2022). Governing the environment-related data space. SSRN Electron. J . doi:10.2139/ssrn.4250166

Greene, S., and Meixell, B. (2017). Hacking the sustainable development goals. Can US cities measure up? [Research report] . Washington, D.C.: Urban Insitute . Available at: https://www.urban.org/sites/default/files/publication/92946/2001490-hacking-the-sustainable-development-goals-corrected.pdf .

Hecker, S., Wicke, N., Haklay, M., and Bonn, A. (2019). How does policy conceptualise citizen science? A qualitative content analysis of international policy documents. Citiz. Sci. Theory Pract. 4 (1), 32. doi:10.5334/cstp.230

Herzog, C., Handke, C., and Hitters, E. (2019). “Analyzing talk and text II: thematic analysis,” in The palgrave handbook of methods for media policy research . Editors H. Van Den Bulck, M. Puppis, K. Donders, and L. Van Audenhove ( Springer International Publishing ), 385–401. doi:10.1007/978-3-030-16065-4_22

Higgins, K., and Cornforth, J. (2015). Civil society and SDG monitoring. Harnessing civil society and citizen-generated data. Datashift. Available at: http://civicus.org/images/SDG%20Monitoring%20-%20DataShift%20Background%20Note%20-%20Final.pdf .

IEAG (2014). Mobilising the data revolution for sustainable development . Independent Expert Advisory Group , A World that Counts. Available at: https://www.undatarevolution.org/wp-content/uploads/2014/11/A-World-That-Counts.pdf .

Imran, A., and Agrawal, R. (2017). “Data provenance,” in Encyclopedia of big data . Editors L. A. Schintler, and C. L. McNeely ( Springer International Publishing ), 1–4. doi:10.1007/978-3-319-32001-4_58-1

Koedel, U., Schuetze, C., Fischer, P., Bussmann, I., Sauer, P. K., Nixdorf, E., et al. (2022). Challenges in the evaluation of observational data trustworthiness from a data producers viewpoint (FAIR+). Front. Environ. Sci. 9, 772666. doi:10.3389/fenvs.2021.772666

Lafortune, G., Fuller, G., Schmidt-Traub, G., Zoeteman, K., Mulder, R., and Dagevos, J. (2019). The 2019 SDG index and dashboards report for European cities . Tilburg: Sustainable Development Solutions Network (SDSN) & Brabant Center for Sustainable Development (Telos, Tilburg University . (prototype version) [Prototype Version]. Available at: https://s3.amazonaws.com/sustainabledevelopment.report/2019/2019_sdg_index_euro_cities.pdf .

Lämmerhirt, D., Jameson, S., and Prasetyo, E. (2016). Making Citizen-Generated Data Work: towards a framework strengthening collaborations between citizens, civil society organisation, and others . Mechelen: Data Shift & Open Knowledge Institute . Available at: https://www.researchgate.net/publication/311654834_Making_Citizen-Generated_Data_Work_Towards_a_framework_strengthening_collaborations_between_citizens_civil_society_organisation_and_others#fullTextFileContent .

MacFeely, S. (2019). The big (data) bang: opportunities and challenges for compiling SDG indicators. Glob. Policy 10 (S1), 121–133. doi:10.1111/1758-5899.12595

Mahajan, S., Chung, M.-K., Martinez, J., Olaya, Y., Helbing, D., and Chen, L.-J. (2022). Translating citizen-generated air quality data into evidence for shaping policy. Humanit. Soc. Sci. Commun. 9 (1), 122. doi:10.1057/s41599-022-01135-2

McKinley, D. C., Miller-Rushing, A. J., Ballard, H. L., Bonney, R., Brown, H., Cook-Patton, S. C., et al. (2017). Citizen science can improve conservation science, natural resource management, and environmental protection. Biol. Conserv. 208, 15–28. doi:10.1016/j.biocon.2016.05.015

Moczek, N., Voigt-Heucke, S. L., Mortega, K. G., Fabó Cartas, C., and Knobloch, J. (2021). A self-assessment of European citizen science projects on their contribution to the UN sustainable development goals (SDGs). Sustainability 13 (4), 1774. doi:10.3390/su13041774

Müller, M., Lorenz, J., Voigt-Heucke, S., Heinrich, G., and Oesterheld, M. (2023). Citizen science for the sustainable development goals? The perspective of German citizen science practitioners on the relationship between citizen science and the sustainable development goals. Citiz. Sci. Theory Pract. 8 (1), 34. doi:10.5334/cstp.583

Nascimento, S., Iglesias, J., Owen, R., Schade, S., and Shanley, L. (2018). “Citizen science for policy formulation and implementation,” in Citizen science: innovation in open science (London: Society and Policy ). Available at: https://www.researchgate.net/publication/330976243_Citizen_science_for_policy_formulation_and_implementation_Innovation_in_Open_Science_Society_and_Policy .

OECD (2020). A territorial approach to the sustainable development goals: synthesis report. OECD . doi:10.1787/e86fa715-en

Oosterhof, P. D. (2018). Localizing the SDGs to Accelerate the Implementation of the 2030 Agenda for sustainable development: the current State of sustainable development goal Localization in Asia and the pacific (0 ed., governance briefs) [governance briefs] . Manila: Asian Development Bank . doi:10.22617/BRF189612

Phillips, T. B., Parker, A., Bowser, A., and Haklay, M. (2021). “Publicly generated data: the role of citizen science for knowledge production, action, and public engagement,”. Closing the knowledge-implementation gap in conservation science . Editors C. C. Ferreira, and C. F. C. Klütsch ( Springer International Publishing ), 4, 83–107. doi:10.1007/978-3-030-81085-6_4

Reddy, P. (2016). Localising the sustainable development goals (SDGs): the role of local government in context. Afr. J. Public Aff. Vol. 9 (Number 2). Available at: https://repository.up.ac.za/bitstream/handle/2263/58190/Reddy_Localising_2016.pdf?sequence=1&isAllowed=y .

San Llorente Capdevila, A., Kokimova, A., Sinha Ray, S., Avellán, T., Kim, J., and Kirschke, S. (2020). Success factors for citizen science projects in water quality monitoring. Sci. Total Environ. 728, 137843. doi:10.1016/j.scitotenv.2020.137843

Sauermann, H., Vohland, K., Antoniou, V., Balázs, B., Göbel, C., Karatzas, K., et al. (2020). Citizen science and sustainability transitions. Res. Policy 49 (5), 103978. doi:10.1016/j.respol.2020.103978

Schleicher, K., and Schmidt, C. (2020). Citizen science in Germany as research and sustainability education: analysis of the main forms and foci and its relation to the sustainable development goals. Sustainability 12 (15), 6044. doi:10.3390/su12156044

SDSN, & IEEP (2019). The 2019 europe sustainable development report: towards a strategy for achieving the sustainable development goals in the European union . Paris and Brussels: Sustainable Development Solutions Network and Institute for European Environmental Policy . Available at: https://ieep.eu/wp-content/uploads/2022/12/Europe-Sustainable-Development-Report-2019.pdf .

Serrano, F., Sanz, F., Pelacho, M., Turbé, A., Barba, J., Mudgal, S., et al. (2018). Citizen science for environmental policy: development of an EU wide inventory and analysis of selected practices . Luxemburg: Publications Office . Available at: https://data.europa.eu/doi/10.2779/961304 .

Sharifi, A., Allam, Z., Bibri, S. E., and Khavarian-Garmsir, A. R. (2024). Smart cities and sustainable development goals (SDGs): a systematic literature review of co-benefits and trade-offs. Cities 146, 104659. doi:10.1016/j.cities.2023.104659

UNDG (2014). Localizing the post-2015 development agenda: dialogues on implementation . New York: United Nations Development Group . Available at: https://uclg.org/sites/default/files/dialogues_on_localizing_the_post-2015_development_agenda.pdf .

UNEP (2021). Measuring progress: environment and the SDGs . Nairobi: United Nations . doi:10.18356/9789210011211

United Nations (2016). The sustainable development goals report 2016 . United Nations. Available at: https://unstats.un.org/sdgs/report/2016/the%20sustainable%20development%20goals%20report%202016.pdf .

United Nations (2020). Measuring and monitoring progress towards the sustainable development goals . United Nations.

Van Audenhove, L., and Donders, K. (2019). “Talking to people III: expert interviews and elite interviews,” in The palgrave handbook of methods for media policy research . Editors M. Puppis, H. Van den Bulck, L. Van Audenhove, and K. Donders ( Palgrave Macmillan ), 179–197. Available at: https://link.springer.com/chapter/10.1007/978-3-030-16065-4_10 .

van den Homberg, M., and Susha, I. (2018). Characterizing data ecosystems to support official statistics with open mapping data for reporting on sustainable development goals. ISPRS Int. J. Geo-Information 7 (12), 456. doi:10.3390/ijgi7120456

West, S., and Pateman, R. (2017). How could citizen science support the sustainable development goals? [Policy brief] . Stockholm: Stockholm Environment Institute . Available at: https://mediamanager.sei.org/documents/Publications/SEI-2017-PB-citizen-science-sdgs.pdf .

Wilkinson, M. D., Dumontier, M., Aalbersberg, Ij. J., Appleton, G., Axton, M., Baak, A., et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data 3 (1), 160018. doi:10.1038/sdata.2016.18

Keywords: smart & sustainable cities, data-driven policymaking, non-traditional data, sustainable development goals, environmental SDG monitoring, indicators, expert interviews, citizen science data

Citation: Borghys K, Vandercruysse L, Veeckman C, Temmerman L and Heyman R (2024) Localizing the sustainable development goals in smart and sustainable cities: how can citizen-generated data support the local monitoring of SDGs? A case study of the Brussels Capital Region. Front. Environ. Sci. 12:1369001. doi: 10.3389/fenvs.2024.1369001

Received: 11 January 2024; Accepted: 11 March 2024; Published: 02 April 2024.

Reviewed by:

Copyright © 2024 Borghys, Vandercruysse, Veeckman, Temmerman and Heyman. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Koen Borghys, [email protected]

This article is part of the Research Topic

Citizen Science and Climate Services in Cities: Current State, New Approaches and Future Avenues for Enhancing Urban Climate Resilience

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

  • Advanced Search
  • Journal List
  • Perspect Med Educ
  • v.8(4); 2019 Aug

Logo of pmeded

Limited by our limitations

Paula t. ross.

Medical School, University of Michigan, Ann Arbor, MI USA

Nikki L. Bibler Zaidi

Study limitations represent weaknesses within a research design that may influence outcomes and conclusions of the research. Researchers have an obligation to the academic community to present complete and honest limitations of a presented study. Too often, authors use generic descriptions to describe study limitations. Including redundant or irrelevant limitations is an ineffective use of the already limited word count. A meaningful presentation of study limitations should describe the potential limitation, explain the implication of the limitation, provide possible alternative approaches, and describe steps taken to mitigate the limitation. This includes placing research findings within their proper context to ensure readers do not overemphasize or minimize findings. A more complete presentation will enrich the readers’ understanding of the study’s limitations and support future investigation.

Introduction

Regardless of the format scholarship assumes, from qualitative research to clinical trials, all studies have limitations. Limitations represent weaknesses within the study that may influence outcomes and conclusions of the research. The goal of presenting limitations is to provide meaningful information to the reader; however, too often, limitations in medical education articles are overlooked or reduced to simplistic and minimally relevant themes (e.g., single institution study, use of self-reported data, or small sample size) [ 1 ]. This issue is prominent in other fields of inquiry in medicine as well. For example, despite the clinical implications, medical studies often fail to discuss how limitations could have affected the study findings and interpretations [ 2 ]. Further, observational research often fails to remind readers of the fundamental limitation inherent in the study design, which is the inability to attribute causation [ 3 ]. By reporting generic limitations or omitting them altogether, researchers miss opportunities to fully communicate the relevance of their work, illustrate how their work advances a larger field under study, and suggest potential areas for further investigation.

Goals of presenting limitations

Medical education scholarship should provide empirical evidence that deepens our knowledge and understanding of education [ 4 , 5 ], informs educational practice and process, [ 6 , 7 ] and serves as a forum for educating other researchers [ 8 ]. Providing study limitations is indeed an important part of this scholarly process. Without them, research consumers are pressed to fully grasp the potential exclusion areas or other biases that may affect the results and conclusions provided [ 9 ]. Study limitations should leave the reader thinking about opportunities to engage in prospective improvements [ 9 – 11 ] by presenting gaps in the current research and extant literature, thereby cultivating other researchers’ curiosity and interest in expanding the line of scholarly inquiry [ 9 ].

Presenting study limitations is also an ethical element of scientific inquiry [ 12 ]. It ensures transparency of both the research and the researchers [ 10 , 13 , 14 ], as well as provides transferability [ 15 ] and reproducibility of methods. Presenting limitations also supports proper interpretation and validity of the findings [ 16 ]. A study’s limitations should place research findings within their proper context to ensure readers are fully able to discern the credibility of a study’s conclusion, and can generalize findings appropriately [ 16 ].

Why some authors may fail to present limitations

As Price and Murnan [ 8 ] note, there may be overriding reasons why researchers do not sufficiently report the limitations of their study. For example, authors may not fully understand the importance and implications of their study’s limitations or assume that not discussing them may increase the likelihood of publication. Word limits imposed by journals may also prevent authors from providing thorough descriptions of their study’s limitations [ 17 ]. Still another possible reason for excluding limitations is a diffusion of responsibility in which some authors may incorrectly assume that the journal editor is responsible for identifying limitations. Regardless of reason or intent, researchers have an obligation to the academic community to present complete and honest study limitations.

A guide to presenting limitations

The presentation of limitations should describe the potential limitations, explain the implication of the limitations, provide possible alternative approaches, and describe steps taken to mitigate the limitations. Too often, authors only list the potential limitations, without including these other important elements.

Describe the limitations

When describing limitations authors should identify the limitation type to clearly introduce the limitation and specify the origin of the limitation. This helps to ensure readers are able to interpret and generalize findings appropriately. Here we outline various limitation types that can occur at different stages of the research process.

Study design

Some study limitations originate from conscious choices made by the researcher (also known as delimitations) to narrow the scope of the study [ 1 , 8 , 18 ]. For example, the researcher may have designed the study for a particular age group, sex, race, ethnicity, geographically defined region, or some other attribute that would limit to whom the findings can be generalized. Such delimitations involve conscious exclusionary and inclusionary decisions made during the development of the study plan, which may represent a systematic bias intentionally introduced into the study design or instrument by the researcher [ 8 ]. The clear description and delineation of delimitations and limitations will assist editors and reviewers in understanding any methodological issues.

Data collection

Study limitations can also be introduced during data collection. An unintentional consequence of human subjects research is the potential of the researcher to influence how participants respond to their questions. Even when appropriate methods for sampling have been employed, some studies remain limited by the use of data collected only from participants who decided to enrol in the study (self-selection bias) [ 11 , 19 ]. In some cases, participants may provide biased input by responding to questions they believe are favourable to the researcher rather than their authentic response (social desirability bias) [ 20 – 22 ]. Participants may influence the data collected by changing their behaviour when they are knowingly being observed (Hawthorne effect) [ 23 ]. Researchers—in their role as an observer—may also bias the data they collect by allowing a first impression of the participant to be influenced by a single characteristic or impression of another characteristic either unfavourably (horns effect) or favourably (halo effort) [ 24 ].

Data analysis

Study limitations may arise as a consequence of the type of statistical analysis performed. Some studies may not follow the basic tenets of inferential statistical analyses when they use convenience sampling (i.e. non-probability sampling) rather than employing probability sampling from a target population [ 19 ]. Another limitation that can arise during statistical analyses occurs when studies employ unplanned post-hoc data analyses that were not specified before the initial analysis [ 25 ]. Unplanned post-hoc analysis may lead to statistical relationships that suggest associations but are no more than coincidental findings [ 23 ]. Therefore, when unplanned post-hoc analyses are conducted, this should be clearly stated to allow the reader to make proper interpretation and conclusions—especially when only a subset of the original sample is investigated [ 23 ].

Study results

The limitations of any research study will be rooted in the validity of its results—specifically threats to internal or external validity [ 8 ]. Internal validity refers to reliability or accuracy of the study results [ 26 ], while external validity pertains to the generalizability of results from the study’s sample to the larger, target population [ 8 ].

Examples of threats to internal validity include: effects of events external to the study (history), changes in participants due to time instead of the studied effect (maturation), systematic reduction in participants related to a feature of the study (attrition), changes in participant responses due to repeatedly measuring participants (testing effect), modifications to the instrument (instrumentality) and selecting participants based on extreme scores that will regress towards the mean in repeat tests (regression to the mean) [ 27 ].

Threats to external validity include factors that might inhibit generalizability of results from the study’s sample to the larger, target population [ 8 , 27 ]. External validity is challenged when results from a study cannot be generalized to its larger population or to similar populations in terms of the context, setting, participants and time [ 18 ]. Therefore, limitations should be made transparent in the results to inform research consumers of any known or potentially hidden biases that may have affected the study and prevent generalization beyond the study parameters.

Explain the implication(s) of each limitation

Authors should include the potential impact of the limitations (e.g., likelihood, magnitude) [ 13 ] as well as address specific validity implications of the results and subsequent conclusions [ 16 , 28 ]. For example, self-reported data may lead to inaccuracies (e.g. due to social desirability bias) which threatens internal validity [ 19 ]. Even a researcher’s inappropriate attribution to a characteristic or outcome (e.g., stereotyping) can overemphasize (either positively or negatively) unrelated characteristics or outcomes (halo or horns effect) and impact the internal validity [ 24 ]. Participants’ awareness that they are part of a research study can also influence outcomes (Hawthorne effect) and limit external validity of findings [ 23 ]. External validity may also be threatened should the respondents’ propensity for participation be correlated with the substantive topic of study, as data will be biased and not represent the population of interest (self-selection bias) [ 29 ]. Having this explanation helps readers interpret the results and generalize the applicability of the results for their own setting.

Provide potential alternative approaches and explanations

Often, researchers use other studies’ limitations as the first step in formulating new research questions and shaping the next phase of research. Therefore, it is important for readers to understand why potential alternative approaches (e.g. approaches taken by others exploring similar topics) were not taken. In addition to alternative approaches, authors can also present alternative explanations for their own study’s findings [ 13 ]. This information is valuable coming from the researcher because of the direct, relevant experience and insight gained as they conducted the study. The presentation of alternative approaches represents a major contribution to the scholarly community.

Describe steps taken to minimize each limitation

No research design is perfect and free from explicit and implicit biases; however various methods can be employed to minimize the impact of study limitations. Some suggested steps to mitigate or minimize the limitations mentioned above include using neutral questions, randomized response technique, force choice items, or self-administered questionnaires to reduce respondents’ discomfort when answering sensitive questions (social desirability bias) [ 21 ]; using unobtrusive data collection measures (e.g., use of secondary data) that do not require the researcher to be present (Hawthorne effect) [ 11 , 30 ]; using standardized rubrics and objective assessment forms with clearly defined scoring instructions to minimize researcher bias, or making rater adjustments to assessment scores to account for rater tendencies (halo or horns effect) [ 24 ]; or using existing data or control groups (self-selection bias) [ 11 , 30 ]. When appropriate, researchers should provide sufficient evidence that demonstrates the steps taken to mitigate limitations as part of their study design [ 13 ].

In conclusion, authors may be limiting the impact of their research by neglecting or providing abbreviated and generic limitations. We present several examples of limitations to consider; however, this should not be considered an exhaustive list nor should these examples be added to the growing list of generic and overused limitations. Instead, careful thought should go into presenting limitations after research has concluded and the major findings have been described. Limitations help focus the reader on key findings, therefore it is important to only address the most salient limitations of the study [ 17 , 28 ] related to the specific research problem, not general limitations of most studies [ 1 ]. It is important not to minimize the limitations of study design or results. Rather, results, including their limitations, must help readers draw connections between current research and the extant literature.

The quality and rigor of our research is largely defined by our limitations [ 31 ]. In fact, one of the top reasons reviewers report recommending acceptance of medical education research manuscripts involves limitations—specifically how the study’s interpretation accounts for its limitations [ 32 ]. Therefore, it is not only best for authors to acknowledge their study’s limitations rather than to have them identified by an editor or reviewer, but proper framing and presentation of limitations can actually increase the likelihood of acceptance. Perhaps, these issues could be ameliorated if academic and research organizations adopted policies and/or expectations to guide authors in proper description of limitations.

IMAGES

  1. advantages and disadvantages of case study in qualitative research

    limitations of a qualitative case study

  2. Qualitative Research: Definition, Methodology, Limitation, Examples

    limitations of a qualitative case study

  3. PPT

    limitations of a qualitative case study

  4. PPT

    limitations of a qualitative case study

  5. PPT

    limitations of a qualitative case study

  6. case study method of qualitative research

    limitations of a qualitative case study

VIDEO

  1. Lecture 46: Qualitative Resarch

  2. Lecture 49: Qualitative Resarch

  3. Lecture 47: Qualitative Resarch

  4. Lecture 50: Qualitative Resarch

  5. Quantitative Research: Its Characteristics, Strengths, and Weaknesses

  6. What is a "Case Study"? Explained!

COMMENTS

  1. Case Study Methodology of Qualitative Research: Key Attributes and

    A case study is one of the most commonly used methodologies of social research. This article attempts to look into the various dimensions of a case study research strategy, the different epistemological strands which determine the particular case study type and approach adopted in the field, discusses the factors which can enhance the effectiveness of a case study research, and the debate ...

  2. Methodology or method? A critical review of qualitative case study

    In addition, discussion about case study limitations has led some authors to query whether case study is indeed a methodology (Luck, Jackson, & Usher, 2006; Meyer, 2001; Thomas, 2010; Tight, 2010). Methodological discussion of qualitative case study research is timely, and a review is required to analyse and understand how this methodology is ...

  3. The Advantages and Limitations of Single Case Study Analysis

    Limitations. Single case study analysis has, however, been subject to a number of criticisms, the most common of which concern the inter-related issues of methodological rigour, researcher subjectivity, and external validity. ... 'Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options', Political ...

  4. (PDF) The case study as a type of qualitative research

    Abstract. This article presents the case study as a type of qualitative research. Its aim is to give a detailed description of a case study - its definition, some classifications, and several ...

  5. Presenting and Evaluating Qualitative Research

    STRENGTHS AND LIMITATIONS OF QUALITATIVE RESEARCH. Qualitative researchers have been criticized for overusing interviews and focus groups at the expense of other methods such as ethnography, observation, documentary analysis, case studies, and conversational analysis. Qualitative research has numerous strengths when properly conducted.

  6. LibGuides: Qualitative study design: Case Studies

    An example of a qualitative case study is a life history which is the story of one specific person. A case study may be done to highlight a specific issue by telling a story of one person or one group. ... Limitations. Labour intensive and generates large diverse data sets which can be hard to manage.

  7. Case Study Research Method in Psychology

    Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews). The case study research method originated in clinical medicine (the case history, i.e., the patient's personal history). In psychology, case studies are ...

  8. Case Study

    The definitions of case study evolved over a period of time. Case study is defined as "a systematic inquiry into an event or a set of related events which aims to describe and explain the phenomenon of interest" (Bromley, 1990).Stoecker defined a case study as an "intensive research in which interpretations are given based on observable concrete interconnections between actual properties ...

  9. Case Study

    A case study is a research method that involves an in-depth examination and analysis of a particular phenomenon or case, such as an individual, organization, community, event, or situation. It is a qualitative research approach that aims to provide a detailed and comprehensive understanding of the case being studied.

  10. The case study approach

    A case study is a research approach that is used to generate an in-depth, multi-faceted understanding of a complex issue in its real-life context. It is an established research design that is used extensively in a wide variety of disciplines, particularly in the social sciences. A case study can be defined in a variety of ways (Table 5 ), the ...

  11. Qualitative Study

    Qualitative research is a type of research that explores and provides deeper insights into real-world problems.[1] Instead of collecting numerical data points or intervene or introduce treatments just like in quantitative research, qualitative research helps generate hypotheses as well as further investigate and understand quantitative data. Qualitative research gathers participants ...

  12. Tools for assessing the methodological limitations of a QES—a short

    The increasing prevalence and application of qualitative evidence syntheses (QES) in decision-making processes underscore the need for robust tools to assess the methodological limitations of a completed QES. This commentary discusses the limitations of three existing tools and presents the authors' efforts to address this gap. Through a simple comparative analysis, the three tools are ...

  13. Strengths and Limitations of Qualitative and Quantitative Research Methods

    Scientific research adopts qualitati ve and quantitative methodologies in the modeling. and analysis of numerous phenomena. The qualitative methodology intends to. understand a complex reality and ...

  14. Ethical Dilemmas in Qualitative Research: A Critical Literature Review

    To reflect on the impasses experienced by the researcher in qualitative research: Case study—ethnography and interview: The attitude of the researcher in the field impacts people's lives, demanding from him a constant ethical question. ... and to know how to face one's own limitations (Bishop & Shepherd, 2011; Guillemin & Guillan, 2015 ...

  15. Limitations of the Study

    Sample Size Limitations in Qualitative Research. Sample sizes are typically smaller in qualitative research because, as the study goes on, acquiring more data does not necessarily lead to more information. This is because one occurrence of a piece of data, or a code, is all that is necessary to ensure that it becomes part of the analysis framework.

  16. How to Write Limitations of the Study (with examples)

    Common types of limitations and their ramifications include: Theoretical: limits the scope, depth, or applicability of a study. Methodological: limits the quality, quantity, or diversity of the data. Empirical: limits the representativeness, validity, or reliability of the data. Analytical: limits the accuracy, completeness, or significance of ...

  17. Methodology or method? A critical review of qualitative case study reports

    Current methodological issues in qualitative case study research. The future of qualitative research will be influenced and constructed by the way research is conducted, and by what is reviewed and published in academic journals (Morse, Citation 2011).If case study research is to further develop as a principal qualitative methodological approach, and make a valued contribution to the field of ...

  18. 10 Case Study Advantages and Disadvantages (2024)

    A case study in academic research is a detailed and in-depth examination of a specific instance or event, generally conducted through a qualitative approach to data. The most common case study definition that I come across is is Robert K. Yin's (2003, p.

  19. Frontiers

    The study focusses on the BCR and involves qualitative data collection through 10 in-depth semi-structured interviews with citizen science data experts from various backgrounds. We started restructuring the original five dimensions of the conceptual model into two overarching categories—data scope and organizational data governance—which ...

  20. Minerals

    Case study results from the Illinois basin demonstrate the effectiveness and applicability of this approach, to assess the risk associated with legacy wells within the AoR of potential CO 2 storage site, strategically categorizing over 4000 such wells despite data limitations.

  21. The case study approach

    A case study is a research approach that is used to generate an in-depth, multi-faceted understanding of a complex issue in its real-life context. It is an established research design that is used extensively in a wide variety of disciplines, particularly in the social sciences. A case study can be defined in a variety of ways (Table.

  22. Limited by our limitations

    Abstract. Study limitations represent weaknesses within a research design that may influence outcomes and conclusions of the research. Researchers have an obligation to the academic community to present complete and honest limitations of a presented study. Too often, authors use generic descriptions to describe study limitations.

  23. How can an integrated multi-method geophysical approach optimise the

    However, modelling this hydrosystem has its limitations due to the spatial and qualitative heterogeneity of the data available. The integration of geophysical data using various methods and scales has improved the geological knowledge by identifying the different chalk formation and highlighting the geometry of the Seine fault system near Rouen.