education research problem solving

Change Password

Your password must have 8 characters or more and contain 3 of the following:.

• a lower case character, 
• an upper case character, 
• a special character 

Password Changed Successfully

Your password has been changed

• Sign in / Register

Request Username

Can't sign in? Forgot your username?

Enter your email address below and we will send you your username

If the address matches an existing account you will receive an email with instructions to retrieve your username

Teaching Discipline-Based Problem Solving

• Regina F. Frey
• Cynthia J. Brame
• Angela Fink
• Paula P. Lemons

Department of Chemistry, University of Utah, Salt Lake City, UT 84112

Search for more papers by this author

Center for Teaching and Department of Biological Sciences, Vanderbilt University, Nashville, TN 37203

Center for Integrative Research on Cognition, Learning, and Education, Washington University in St. Louis, St. Louis, MO 63130-4899

*Address correspondence to: Paula P. Lemons ( E-mail Address: [email protected] ).

Department of Biochemistry & Molecular Biology, University of Georgia, Athens, GA 30602

Problem solving plays an essential role in all scientific disciplines, and solving problems can reveal essential concepts that underlie those disciplines. Thus, problem solving serves both as a common tool and desired outcome in many science classes. Research on teaching problem solving offers principles for instruction that are guided by learning theories. This essay describes an online, evidence-based teaching guide ( https://lse.ascb.org/
evidence-based-teaching-guides/problem-solving ) intended to guide instructors in the use of these principles. The guide describes the theoretical underpinnings of problem-solving research and instructional choices that can place instruction before problem solving (e.g., peer-led team learning and worked examples) or problem solving before instruction (e.g., process-oriented guided inquiry learning, contrasting cases, and productive failure). Finally, the guide describes assessment choices that help instructors consider alternative outcomes for problem-solving instruction. Each of these sections consists of key points that can be gleaned from the literature as well as summaries and links to articles that inform these points. The guide also includes an instructor checklist that offers a concise summary of key points with actionable steps to direct instructors as they develop and refine their problem-solving instruction.

INTRODUCTION

Recent calls for reform in undergraduate science education such as Vision and Change and the Next Generation Science Standards focus on the importance of teaching core concepts and scientific practices ( American Association for the Advancement of Science, 2011 ; NGSS, 2013 ). Core concepts are the principles that undergird disciplinary knowledge in the sciences, and scientific practices are the ways scientists go about using these disciplinary concepts in authentic practice. Problem solving fits nicely into these reform frameworks, because problem solving is a key scientific practice, yet it also provides a mechanism by which students obtain deep conceptual understanding. For example, imagine a student who has been asked to solve a problem about the functional differences in the spike protein mutations of SAR-CoV2 variants. In solving this problem, the student will be deepening conceptual understanding while simultaneously practicing science.

Problem solving occurs when people attempt a task for which the path to completing that task is uncertain ( Martinez, 1998 ). Problems are the tasks themselves. Problem solving occurs in everyday life and can involve everything from deciding the route to a new location to determining the best way to approach a complex work challenge. In the academic setting, problem solving typically pertains to the challenge of solving discipline-based problems. These problems may be authentic to professional work (e.g., determining the appropriate analyses for a research data set) or related to the concepts and procedures that comprise a body of disciplinary knowledge (e.g., solving a steady-state problem in biochemistry).

When thinking of academic problem solving, many people automatically think of disciplines that incorporate problem types that can be solved using algorithms and heuristics. For example, mathematics, chemistry, physics, and genetics all include problems that can be solved with equations or other quantitative tools. Yet all disciplines involve problem solving, because all disciplines consist of foundational concepts that undergird the discourse and work of that discipline. For example, professional biologists conduct their research and think about their results guided by the concepts of evolution by natural selection and genetic drift, while professional biochemists rely on the concept of structure and function to guide research and development.

The aim of this evidence-based teaching guided (EBTG) is to present the evidence-base for teaching problem solving ( Figure 1 ), and readers will benefit from considering four key advances in problem-solving research. More complete reviews of the history of problem-solving research have been provided by others (e.g., Bassok and Novick, 2012 ). First, the Gestalt psychologists of the 1940s investigated problem representation, that is, how people construct a model of the problem that summarizes their understanding of the problem. They found that visual aspects of problems and a solver’s prior knowledge affect problem representation and, in turn, how a solver generates solutions (e.g., Duncker, 1945 ; Polya, 1957 ; Wertheimer, 1959 ; Bassok and Novick, 2012 ). Second, seminal work in the 1970s focused on the problem-solving process, theorizing problem solving as a search of the problem space for a path that connects the initial state to the solution. Researchers identified general-purpose problem-solving strategies, such as brainstorming and working backward ( Polya, 1962 ; Simon and Newell, 1972 ; Jonassen, 2000 ). Third, work in the later 1970s and 1980s shifted away from puzzle problems to focus on knowledge-rich domains, including chess, mathematics, and physics ( Chase and Simon, 1973 ; Hinsley et al. , 1978 ; Larkin et al. , 1980 ; Chi et al. , 1981 ; Gobert and Simon, 1996 ). This research confirmed the importance of prior knowledge in problem solving and crystallized the fundamental differences in knowledge organization and processing between experts and novices. Fourth, modern problem-solving research returns to the question of problem representation, incorporating new research on visual processing to clarify how problem presentation interacts with students’ prior knowledge in a specific academic discipline ( Novick and Catley, 2007 , 2014 ; Goldstone et al. , 2010 ; Fisher et al. , 2011 ). Moreover, researchers have studied how to design instructional experiences in ways that support basic principles of human problem solving. They have asked the question: Given what we know about cognitive architecture and processing, how should we teach problem solving, and how can problem solving itself become an activity to facilitate learning?

This EBTG focuses on the evidence base for teaching problem solving. Our aim is to help instructors make decisions about teaching and assessing problem solving within their disciplines. We start by presenting the relevant theories that guided the development of problem-solving pedagogies, and then move to instructional choices. We focus on two key approaches that differ based on the sequence of events: instruction followed by problem solving (hereafter referred to as I→PS) and problem solving followed by instruction (hereafter referred to as PS→I). A strong evidence base exists for both approaches. We also consider the choices an instructor must make about assessing problem solving. The level of challenge of any given assessment depends on the knowledge of the person undergoing the assessment; problem-solving researchers have historically dealt with this issue using the concept of transfer. Transfer refers to the application of facts, procedures, or concepts to a new problem. Finally, we conclude by pointing out several important research questions that still need to be answered.

Note that this guide intends to synthesize our knowledge surrounding problem solving, which largely derives from the educational psychology literature. We have selected a subset of pedagogical approaches that fit within this framework and are well supported in the literature. In some cases, these approaches have been studied in undergraduate biology classes, and we have cited those papers. In other cases, the approaches have not been studied in undergraduate biology, so we drew upon research in the context of chemistry, physics, and statistics undergraduate education, as well as K–12 settings. Even though these contexts are different, we think readers can draw connections to see how they might use and test these approaches in their biology courses.

FIGURE 1. Problem solving guide landing page

THEORETICAL UNDERPINNINGS

Constructivism is the foundation for all instructional practices that focus on active problem solving. Pedagogies for problem solving also draw on additional theories about learning, including cognitive load theory; activating, differentiating, and encoding knowledge; desirable difficulties; preparation for future learning; and learner agency.

Constructivism

A range of instructional approaches (e.g., inquiry learning, problem-based learning, case-based methods, and collaborative/cooperative methods; Prince and Felder, 2006 ) are all described as using a constructivist framework. Having such varied approaches can make it difficult for instructors to obtain a complete view of the constructivist theory and how it applies to these instructional approaches.

Broadly defined, constructivist theory asserts that students learn (i.e., construct mental models) by integrating new information into their prior knowledge ( Bodner, 1986 ; Driver et al. , 1994 ). That is, students need to construct their own versions of a phenomenon based on observations or data instead of simply absorbing a version presented to them by an instructor or more advanced peers. Hence, students cannot be passive during the learning process. They must actively select information to integrate into their existing knowledge and continually test their mental model by solving problems, asking and answering questions, and discussing their ideas. Two broad categories exist in constructivist theory (reviewed by Amineh and Asl, 2015 ). One category aligns with Piaget’s work on cognitive development. This category focuses on the individual learner’s construction of knowledge to enhance or modify existing mental models. The second category aligns with Vygotsky’s work emphasizing the importance of social, cultural, and historical influences on learners’ knowledge construction. Vygotskian or social constructivism assumes that students’ understanding of new information, their evaluation of its importance, and their sense-making of the information develop in collaboration with other learners.

Cognitive Theories and Frameworks

In addition to constructivism, other cognitive theories and conceptual frameworks underlie much of the research presented in this EBTG. One prominent theory initially described by Sweller (1988) is cognitive load theory (CLT). The basic premise concerns the level of cognitive load, or demand, that learning activities place on students’ working memory. Sweller (1988) proposed that, during problem-solving activities, experts use existing mental schemas (i.e., long-term memory structures) to categorize features and move forward in the problem-solving process. Novices, however, do not have these robust existing schemas and must use cognitively demanding general strategies to try many different pathways to solve the problem. CLT explains that students learning new information must therefore rely heavily on working memory, which has limited capacity, and this high cognitive demand can impede learning underlying problem features. I→PS approaches address this issue by introducing students to new concepts before asking them to actively apply the concepts during problem solving. Providing instruction first arguably helps students transfer the information to long-term memory, which is not capacity limited like working memory, and results in a low cognitive load. When students subsequently solve problems, they can retrieve the information from long-term memory and dedicate their working memory resources to detecting underlying problem features. Extensive research supports CLT and the effect of limited working memory capacity on student learning ( Sweller et al. , 2019 ).

Several other cognitive theories and conceptual frameworks can serve as alternative lenses to understand instruction that uses and fosters problem solving. For instance, rather than focusing on the limitations of working memory, instructors might consider how to help students activate, differentiate, and encode knowledge. As noted earlier, learning requires integrating new information into prior knowledge (e.g., models for phenomena) and addressing any conflicts between the two. PS→I approaches can facilitate this process by prompting students to activate their current mental models, find similarities and differences with their prior experiences and the new information presented, and incorporate variation and critical features of the novel problem into refined mental models. In other words, having students explore new problems first, without prior instruction on the underlying principles, can prompt students to engage in abstract processing rather than simply seeking the solution, resulting in better conceptual learning and transfer to new contexts ( DeCaro and Rittle-Johnson, 2012 ). PS→I can be especially beneficial when it invites students to compare cases and invent general principles to explain the observed variation ( Schwartz et al. , 2011 ).

This approach also aligns with the preparation for future learning theory, which emphasizes students’ readiness to interpret and generate understanding from novel resources and activities. Research shows that the exploration and invention before direct instruction that occurs in PS→I approaches can ultimately help students extract more information from novel future materials, preparing them to independently develop and transfer conceptual understanding ( Schwartz and Martin, 2004 ; Belenky and Nokes-Malach, 2012 ).

The concept of desirable difficulties can also be used to understand instruction that promotes the development of problem-solving ability. While students may prefer strategies that provide quick familiarity and an easy sense of understanding (e.g., rereading), the introduction of cognitive challenges into the learning process can generate better long-term learning outcomes ( Schmidt and Bjork, 1992 ). Such desirable difficulties include requiring students to retrieve information from memory to solve a problem (i.e., retrieval practice) and spacing out practice across varying intervals of time (i.e., distributed practice; Dunlosky et al. , 2013 ). In general, activities where students must actively generate and even struggle to find solutions, instead of following a prescribed and known procedure, will encourage deeper processing of the information. Thus, while PS→I approaches (e.g., invention tasks) might introduce difficulties – slower and effortful learning, more errors, and apparent forgetting of information–they offer the long-term benefits of improved retention and transfer.

Affective Dimensions

While most studies presented in this EBTG focus on the cognitive processes underlying problem-solving pedagogies, recent research has begun to consider student perceptions and affect as well. When complex problem solving is designed to engage students in desirable difficulties, the experience of struggle may undermine student motivation and confidence. As a result, problems should be calibrated to student characteristics (e.g., prior knowledge), so they are achievable with support and structure. Instructors must try to cultivate students’ motivation and confidence, so students can cope with or even embrace the difficulties they encounter ( Zepeda et al. , 2020 ).

One approach to improving motivation is to foster learner agency , or a sense of ownership, by providing choices during the problem-solving process ( Zepeda et al. , 2020 ). Students who have agency in their learning are more motivated and engaged, with positive effects on learning outcomes. PS→I approaches encourage students to test out different ways to solve a problem ( DeCaro and Rittle-Johnson, 2012 ), especially when the students are discussing ideas in small groups. Even when this generative process results in incorrect responses, it can instill learner agency and prepare students for self-directed learning in the future ( Schwartz and Martin, 2004 ).

Another motivational benefit of PS→I approaches is the way they direct students’ attention toward growing their understanding (i.e., a learning or mastery goal) rather than simply finding the correct solution (i.e., a performance goal). Students with mastery orientations show better transfer of problem-solving skills, and invention activities before explicit instruction can lead students to adopt such mastery goals, at least in the short term ( Belenky and Nokes-Malach, 2012 ). Such activities immerse students in the process of discovering underlying features and principles, rather than applying a prescribed procedure to solve the problem.

INSTRUCTIONAL CHOICES

The theories and frameworks that can help us understand instruction that fosters problem-solving abilities leave an array of instructional choices. Our guide organizes the literature on choices about the sequencing of instructional activities into I→PS and PS→I. The I→PS section focuses on peer-led team learning and worked examples plus practice. The PS→I section focuses on process-oriented guided inquiry learning, contrasting cases, and productive failure. Other pedagogies exist that could be included in these sections, but we have selected the ones for which a strong evidence base exists and guidelines for implementation are well delineated. Readers should note that not all these pedagogies originated with an eye toward the I→PS versus PS→I distinction. However, contemporary research points out that the literature on problem-solving instruction can be reorganized based on this distinction. Doing so reveals a testable research question that cuts across pedagogies and is not yet fully answered ( Kapur, 2016 ; Loibl et al. , 2017 ): What are the benefits and limitations of instructing first and then asking students to solve problems versus giving them problems to solve and then following with instruction?

Sequencing Instructional Activities: Instruction Followed by Problem Solving

In I→PS, instructors provide explicit instruction to students before asking them to solve problems. The explicit instruction teaches students the procedure for solving a problem or the concepts involved in solving the problem. Two common I→PS approaches include peer-led team learning, which is a form of guided inquiry, and worked examples plus practice.

Peer-Led Team Learning.

Peer-led team learning (PLTL), which originated in undergraduate chemistry education ( Varma-Nelson et al. , 2004 ); ( Wilson and Varma-Nelson, 2016 ) and is popular in undergraduate biology education as well (e.g., ( Preszler, 2009 ; Snyder et al. , 2016 ), involves students working in collaborative groups to solve prepared problems facilitated by a trained peer leader. PLTL is an I→PS approach, because students first receive explicit instruction about concepts and procedures in a traditional lecture and then attend PLTL sessions to collaboratively practice applying the concepts and procedures to a problem set. Course instructors write the weekly problem sets and design them to be collaborative exercises that engage students in reasoning and increase in complexity from start to finish. Peer leaders guide students to solve the problems with their group members ( Repice et al. , 2016 ). PLTL works best when the PLTL sessions are an integral part of the course and course instructors organize the program and train peer leaders ( Varma-Nelson et al. , 2004 ).

PLTL derives from the theory of social constructivism. Students must develop their conceptual understanding and problem-solving ability through active engagement with the material and sense-making done in collaboration with other students. PLTL sessions provide a social constructivist environment, in that students participate in problem solving and conceptual development with their peers, guided by a more knowledgeable peer ( Wilson and Varma-Nelson, 2016 ).

This guide helps readers understand the characteristics and evidence base for PLTL by focusing on articles about key components (e.g., Wilson and Varma-Nelson, 2016 ) and outcomes, including improvements in exam performance and reductions in drop/fail/withdrawal rates that are particularly profound for students from underserved groups (e.g., Frey et al. , 2018 ). Finally, the guide summarizes studies that describe the types of discourse among PLTL groups, showing that groups engage in talking science, sense-making, and authentic scientific practice (e.g., Bierema et al. , 2017 ).

Worked Examples plus Practice.

Worked examples plus practice originated among educational psychologists who attempted to develop an instructional strategy that aligns with CLT (e.g., ( Tuovinen and Sweller, 1999 ). Researchers aimed to create a pedagogy that would enhance support for intrinsic cognitive load, that is, the cognitive load that comes from the inherent challenge of the procedures and concepts to be learned. For example, it is inherently demanding to conceptualize the electrostatic charge around an atom. At the same time, researchers aimed to reduce extraneous cognitive load, that is, the cognitive load that comes from the way procedures and concepts are taught. For example, it is unnecessarily demanding to conceptualize the electrostatic charge around an atom without a model or visual representation. Researchers achieved this feat through the worked examples plus practice pedagogy, which has been tested in a range of disciplines and educational levels ( Atkinson et al. , 2000 ; Kalyuga et al. , 2001 ; Nievelstein et al. , 2013 ; Halmo et al. , 2020 ).

Worked examples plus practice involves leading students through a problem solution in a step-by-step manner followed by a practice problem. Worked examples plus practice is an I→PS approach, in that the worked examples, sometimes accompanied by additional elaboration by the instructor, provide explicit instruction in the procedures and concepts to be used for solving the problem. Then students take time to solve practice problems, presumably applying the procedures and concepts they have seen in the example. Ideally, this approach involves multiple rounds of worked examples followed by practice problems.

Worked examples plus practice can be an effective instructional approach in helping students to solve problems like those used during instruction (i.e., near transfer; see Assessment Choices ). This benefit appears to be limited to novices within a domain ( Kalyuga et al. , 2001 ). Critics of this approach suggest that it may hamper students’ deep conceptual understanding and far-transfer problem solving, arguing that reducing cognitive load short-circuits desirable difficulties that help students recognize and encode the deep principles underlying a challenging problem ( Kapur, 2016 ).

This guide helps readers understand the characteristics of and evidence for worked examples plus practice. Summaries and links to papers from the late 1990s and early 2000s show readers how the approach took shape based on CLT, while more current references provide details about how to implement this approach and present evidence for its efficacy.

Sequencing Instructional Activities: Problem Solving Followed by Instruction

In PS→I, students solve problems before receiving explicit instruction on relevant procedures and concepts. Three common PS→I approaches include process-oriented guided inquiry learning, contrasting cases, and productive failure, and we expand on these approaches in this section. Readers may also be interested in learning more about problem-based learning ( Allen and Tanner, 2003 ; Anderson et al. , 2005 ; Anderson et al. , 2008 ).

Process-Oriented Guided Inquiry Learning.

Process-oriented guided inquiry learning (POGIL) presents students with conceptual models of the material to be learned and a series of questions that walk students through the process of understanding, explaining, and solving problems pertaining to the model ( Moog, 2014 ; Loertscher and Minderhout, 2019 ; Rodriguez et al. , 2020 ). Students work in collaborative groups of three to four students during class time. POGIL instructional materials provide structured guidance prompting students to explore, understand, and apply a conceptual model. The materials also guide students to develop skills in communication, teamwork, management, and critical thinking. Class sessions contain little or no traditional lecture. Rather, explicit instruction is provided as the instructor facilitates collaborative groups and directs and responds to structured, intermittent report-outs by groups to the entire class ( Moog, 2014 ; Rodriguez et al., 2020 ). We present POGIL as a PS→I approach, because problem solving always occurs before explicit instruction, even though the phases of problem solving and instruction are more intermingled than with productive failure and contrasting cases.

POGIL derives from the theory of social constructivism (reviewed by Amineh and Asl, 2015 ), like PLTL. POGIL provides a social constructivist learning environment where students participate with one another in problem solving facilitated by their instructor, the more knowledgeable guide. Collaboration and guidance offer continuous sense-making opportunities whereby students refine their knowledge and build their skills.

The guide summarizes papers that detail how POGIL is implemented and the impacts of POGIL on student learning and achievement, particularly when compared with traditional lecture (e.g., Vincent-Ruz et al. , 2020 ). The guide also summarizes research on the impact of POGIL on student reasoning and discourse (e.g., Moon et al. , 2016 ).

Contrasting Cases.

Contrasting cases are problems that differ in key features. Comparing the cases can help students identify deep features of the problem type and facilitate conceptual understanding ( Schwartz et al. , 2011 ). Contrasting cases have been used in a variety of ways, including before and after direct instruction ( Roelle and Berthold, 2015 ), to prompt students to invent the principle that unites the cases ( Shemwell et al. , 2015 ), and to prompt students to understand experts’ descriptions of the similarities and differences in the cases ( Newman and DeCaro, 2019 ). We present contrasting cases as a PS→I approach, because they are generally found to be more beneficial when used before explicit instruction ( Alfieri et al. , 2013 ). The guide summarizes papers that show the efficacy of contrasting cases for student learning compared with other approaches, including lecture followed by practice problem solving. The guide also summarizes various uses of contrasting cases and the outcomes of these different uses.

Productive Failure.

The productive failure hypothesis suggests that, under certain conditions, solving ill-structured problems that are beyond students’ skill sets and abilities can promote learning, even though failure may initially occur ( Kapur, 2008 ). Thus, in the productive failure approach, students initially attempt complex problems that are beyond their capabilities. Instructors then provide explicit instruction that reveals conceptual knowledge and problem-solving procedures. The guide summarizes seminal papers that explain the rationale and supporting literature for the productive failure approach, as well as studies that document improved learning outcomes for this approach (e.g., Kapur, 2011 ).

Both contrasting cases and productive failure ask students to solve complex, challenging problems. While these approaches to instruction originated and have been investigated independently, a recent review examined the evidence for these two approaches and lumped them under the broader umbrella of PS→I approaches ( Loibl et al. , 2017 ). This review emphasized that both approaches can be effective in promoting conceptual understanding and problem solving if they include one of two features: 1) The initial challenging problem includes contrasting cases, that is, problems that differ in key features and thus draw students’ attention to the nuances of the underlying principles. (2) The explicit instruction phase builds on student work generated during the problem-solving phase. Instructors do this by drawing attention to the ways students attempted to solve the problem and connect those attempts with canonical solutions to the problem ( Loibl et al. , 2017 ).

Additionally, both productive failure and contrasting cases arise from similar theoretical orientations ( Kapur, 2008 ; Schwartz et al. , 2011 ). First, both draw on the notion that learning requires the activation and differentiation of prior knowledge. Learners develop conceptual understanding as they sort out their relevant knowledge and discover gaps and limitations in their prior knowledge. Solving a complex, challenging problem may cause students to activate a wide range of prior knowledge, identify things they need to know but do not know, and begin to differentiate more and less important knowledge. Second, both draw on preparation for future learning, the idea that the main benefit of a learning experience may be that it sets up a student for greater learning in the future ( Schwartz and Martin, 2004 ; Belenky and Nokes-Malach, 2012 ). An initial, challenging problem-solving phase will have fewer short-term gains than explicit instruction, but the effort and cognitive activation and differentiation likely prepare the learner to benefit greatly from subsequent explicit instruction. Third, both draw on the concept of desirable difficulties ( Schmidt and Bjork, 1992 ). Learning requires effort, and effort that is appropriate and well managed can benefit learning in the long run, even though it is more difficult in the short run. Fourth, both focus on the importance of learner agency ( Zepeda et al. , 2020 ), pointing out that challenging students initially and then supporting them more explicitly can build learners’ confidence and support them to take greater responsibility for their learning by teaching them to determine for themselves what they do and do not need to learn.

Assessment Choices

Evidence-based instructional practice involves backward design ( Wiggins and McTighe, 1998 ). Instructors first define the learning objectives and next decide how they will measure students’ accomplishment of the objectives. Instructors whose objective is for students to learn problem solving must also assess problem solving. What is a problem-solving assessment? A problem-solving assessment is simply a problem. It requires students to complete a task for which the solution is unknown in advance ( Martinez, 1998 ). Yet even a problem for which the solution is unknown in advance may be more or less challenging for a student based on the similarity of the problem to instruction. Historically, researchers have dealt with this issue through the concept of transfer.

Transfer describes students’ ability to solve problems that extend beyond the examples that are directly taught. It deals with students’ ability to use knowledge in a new context. The guide introduces readers to a taxonomy of transfer ( Barnett and Ceci, 2002 ). While most science instructors think first about cognitive transfer (e.g., concepts and procedures), the Barnett and Ceci taxonomy provokes broader thinking, proposing nine dimensions along which transfer can be considered, such as knowledge (i.e., the content of a particular field to which the task is to be applied), functional context (e.g., Is the task positioned as an academic activity or a “real-world” activity?), and social context (i.e., Is the task learned and performed individually or in a group?).

Regardless of the dimension of transfer, instructors must consider how near versus far the transfer problem will require students to go from the original learning environment. A problem that is similar to an example encountered in class or homework can be thought of as a near-transfer problem. Because there is a large overlap between the original learning situation and the new problem, solving the new problem requires students to do something like the previous exposure. On the other end of the spectrum, a far-transfer problem involves concepts to which the learner has previously been exposed but cannot solve with a previously used method. Rather, the learner needs to comprehend the underlying concepts and generate solutions, either by applying the concepts in a different manner or integrating across multiple concepts. Far-transfer problems bear little similarity to the original learning situation. McDaniel and colleagues ( 2018 ) and Frey and colleagues ( 2020 ) developed a rubric with specific example problems to show and characterize near- and far-transfer problems in general chemistry.

The guide also considers the issue of problem representation as it pertains to assessment. Recall that problem representation refers to the mental model a solver constructs that summarizes understanding of the problem ( Bassok and Novick, 2012 ). When students learn via problem solving, they form problem representations. Exemplar learners rely extensively on memorization of specific example problems or algorithms to represent the problems they learned, while abstraction learners develop representations that pertain to the underlying concepts of the problems they learned. These different learning approaches result in similar performances on near-transfer problems, but abstraction learners achieve higher performance on far-transfer problems ( McDaniel et al. , 2018 ).

In addition to knowledge transfer being affected by instructional choice, other outcomes (such as affective outcomes) may be influenced, and hence instructors may want to assess these outcomes. For example, invention or problem solving first approaches have been shown to improve motivation ( Belenky and Nokes-Malach, 2012 ) and increase engagement and improve students’ ability to develop multiple solutions ( Taylor et al. , 2010 ). Often self-report surveys, process-oriented rubrics, and observation data are used to assess these outcomes.

Thus, the guide provides information that should help instructors as they develop an assessment plan, determining 1) what outcomes should be assessed by looking at course objectives and examining the range of transfer dimensions available; 2) what degrees of transfer are of interest and how to develop the questions based on these degrees; and 3) whether there are surveys, rubrics, or other types of data that could be used to assess affective or behavioral outcomes.

EMERGING ISSUES IN PROBLEM-SOLVING RESEARCH AND IMPLICATIONS FOR INSTRUCTION

What is the impact of problem-solving pedagogies across different subdisciplines of biology and different topics in biology?

What is the role of guidance during problem-solving instruction? Do students benefit from explicit structures that fade away as they learn, or are there important benefits to leaving students to solve problems without assistance?

What is the role of prior knowledge? Do different instructional approaches work better for students with more limited prior knowledge, while others work better for students with more prior knowledge?

Are there specific problem types or student characteristics that make problem-solving sequencing before or after instruction better or worse?

How can instructors structure their lessons to provide opportunities for transfer? What does it look like for students to practice transfer?

Arguably, problem-solving research should not be limited to measures of transfer that are purely defined by experts. How can we use other methods, such as classroom observations, focus groups, and interviews, to capture the application of learning (i.e., transfer) from the perspective of the student as opposed to the expert?

Problem-solving research historically has focused on cognitive outcomes, yet the affective impacts of instruction certainly make a difference to student learning and development. Do different approaches to teaching and assessing problem solving result in differential impacts on student interest, motivation, self-efficacy, and sense of belonging?

Despite these outstanding questions, instructors who take the time to consider the gathered evidence for problem-solving instruction and assessment, connect the evidence to learning theory, and critically engage with the outstanding questions will be well equipped to examine their own teaching and improve their capacity for teaching problem solving. In addition, these considerations can prompt the development of experiments in and outside the classroom to better understand the affordances and limitations of different approaches to teaching problem solving.

ACKNOWLEDGMENTS

This material is partially based upon work supported by the National Science Foundation under grant no. DRL1350345 awarded to P.P.L. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

• Alfieri, L., Nokes-Malach, T. J., & Schunn, C. D. ( 2013 ). Learning through case comparisons: A meta-analytic review . Educational Psychologist , 48 (2), 87–113. Google Scholar
• Allen, D., & Tanner, K. ( 2003 ). Approaches to cell biology teaching: Learning content in context—problem-based learning . Cell Biology Education , 2 , 73–81. Link ,  Google Scholar
• American Association for the Advancement of Science . ( 2011 ). Vision and change in undergraduate biology education: A call to action . Retrieved February 22, 2022, from http://www.visionandchange.org Google Scholar
• Amineh, R. J., & Asl, H. D. ( 2015 ). Review of constructivism and social constructivism . Journal of Social Sciences, Literature and Languages , 1 (1), 9–16. Google Scholar
• Anderson, W. L., Mitchell, S. M., & Osgood, M. P. ( 2005 ). Comparison of student performance in cooperative learning and traditional lecture-based biochemistry classes . Biochemistry and Molecular Biology Education , 33 , 387–393. Medline ,  Google Scholar
• Ashman, G., Kalyuga, S., & Sweller, J. ( 2020 ). Problem-solving or explicit instruction: Which should go first when element interactivity is high? Educational Psychology Review , 32 (1), 229–247. Google Scholar
• Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. ( 2000 ). Learning from examples: Instructional principles from the worked examples research . Review of Educational Research , 70 (2), 181–214. Google Scholar
• Barnett, S. M., & Ceci, S. J. ( 2002 ). When and where do we apply what we learn? A taxonomy for far transfer . Psychological Bulletin , 128 (4), 612. Medline ,  Google Scholar
• Bassok, M., & Novick, L. R. ( 2012 ). Problem solving . In Holyoak, K. J.Morrison, R. G. (Eds.), Oxford handbook of thinking and reasoning (pp. 413–432). Oxford, UK: Oxford University Press. Google Scholar
• Belenky, D. M., & Nokes-Malach, T. J. ( 2012 ). Motivation and transfer: The role of mastery-approach goals in preparation for future learning . Journal of the Learning Sciences , 21 (3), 399–432. Google Scholar
• Bierema, A. M.-K., Schwarz, C. V., & Stoltzfus, J. R. ( 2017 ). Engaging undergraduate biology students in scientific modeling: Analysis of group interactions, sense-making, and justification . CBE—Life Sciences Education , 16 (4), ar68. Link ,  Google Scholar
• Bodner, G. M. ( 1986 ). Constructivism: A theory of knowledge . Journal of Chemical Education , 63 (10), 873. Google Scholar
• Chase, W. G., & Simon, H. A. ( 1973 ). Perception in chess . Cognitive Psychology , 4 (1), 55–81. Google Scholar
• Chen, O., & Kalyuga, S. ( 2020 ). Exploring factors influencing the effectiveness of explicit instruction first and problem-solving first approaches . European Journal of Psychology of Education , 35 (3), 607–624. Google Scholar
• Chi, M. T. H., Feltovich, P. J., & Glaser, R. ( 1981 ). Categorization and representation of physics problems by experts and novices . Cognitive Science , 5 , 121–152. Google Scholar
• DeCaro, M. S., & Rittle-Johnson, B. ( 2012 ). Exploring mathematics problems prepares children to learn from instruction . Journal of Experimental Child Psychology , 113 (4), 552–568. Medline ,  Google Scholar
• Driver, R., Asoko, H., Leach, J., Scott, P., & Mortimer, E. ( 1994 ). Constructing scientific knowledge in the classroom . Educational Researcher , 23 (7), 5–12. Google Scholar
• Duncker, K. ( 1945 ). On problem-solving Psych Monographs , 58 , 270. Google Scholar
• Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. ( 2013 ). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology . Psychological Science in the Public Interest , 14 (1), 4–58. Medline ,  Google Scholar
• Fisher, K. J., Borchert, K., & Bassok, M. ( 2011 ). Following the standard form: Effects of equation format on algebraic modeling . Memory & Cognition , 39 (3), 502–515. Medline ,  Google Scholar
• Frey, R. F., Fink, A., Cahill, M. J., McDaniel, M. A., & Solomon, E. D. ( 2018 ). Peer-led team learning in general chemistry I: Interactions with identity, academic preparation, and a course-based intervention . Journal of Chemical Education , 95 (12), 2103–2113. Google Scholar
• Frey, R. F., McDaniel, M. A., Bunce, D. M., Cahill, M. J., & Perry, M. D. ( 2020 ). Using students’ concept-building tendencies to better characterize average-performing student learning and problem-solving approaches in general chemistry . CBE—Life Sciences Education , 19 (3), ar42. Link ,  Google Scholar
• Gobert, F., & Simon, H. ( 1996 ). Recall of rapidly presented random chess positions is a function of skill . Psychonomic Bulletin & Review , 3 , 159–163. Medline ,  Google Scholar
• Halmo, S. M., Sensibaugh, C. A., Reinhart, P., Stogniy, O., Fiorella, L., & Lemons, P. P. ( 2020 ). Advancing the guidance debate: Lessons from educational psychology and implications for biochemistry learning . CBE—Life Sciences Education , 19 (3), ar41. Link ,  Google Scholar
• Hinsley, D. A., Hayes, J. R., & Simon, H. A. ( 1978 ). From words to equations: Meaning and representation in algebra word problems . In Carpenter, P. A.Just, M. A. (Eds.), Cognitive processes in comprehension . Hillsdale, NJ: Erlhaum. Google Scholar
• Jonassen, D. H. ( 2000 ). Toward a design theory of problem solving . Educational Technology Research and Development , 48 (4), 63–85. Google Scholar
• Kalyuga, S., Chandler, P., & Sweller, J. ( 2001 ). Learner experience and efficiency of instructional guidance . Educational Psychology , 21 (1), 5–23. Google Scholar
• Kapur, M. ( 2008 ). Productive failure . Cognition and Instruction , 26 (3), 379–424. Google Scholar
• Kapur, M. ( 2011 ). A further study of productive failure in mathematical problem solving: Unpacking the design components . Instructional Science , 39 (4), 561–579. Google Scholar
• Larkin, J. H., McDermott, J., Simon, D. P., & Simon, H. A. ( 1980 ). Models of competence in solving physics problems . Cognitive Science , 4 , 317–345. Google Scholar
• Lobato, J. ( 2012 ). The actor-oriented transfer perspective and its contributions to educational research and practice . Educational Psychologist , 47 (3), 232–247. Google Scholar
• Loertscher, J., & Minderhout, V. ( 2019 ). Implementing guided inquiry in biochemistry: Challenges and opportunities . In Bussey, T. J.Cortes, K. L.Austin, R. C. (Eds.), Biochemistry education: from theory to practice (pp. 111–126).. Washington, DC: American Chemical Society.. Google Scholar
• Loibl, K., Roll, I., & Rummel, N. ( 2017 ). Towards a theory of when and how problem solving followed by instruction supports learning . Educational Psychology Review , 29 (4), 693–715. Google Scholar
• Martinez, M. E. ( 1998 ). What is problem solving? Phi Delta Kappan , 79 (8), 605–609. Google Scholar
• McDaniel, M. A., Cahill, M. J., Frey, R. F., Rauch, M., Doele, J., Ruvolo, D., & Daschbach, M. M. ( 2018 ). Individual differences in learning exemplars versus abstracting rules: Associations with exam performance in college science . Journal of Applied Research in Memory and Cognition , 7 , 241–251. Google Scholar
• Moog, R. ( 2014 ). Process oriented guided inquiry . In McDaniel, M. A.Frey, R. F.Fitzpatrick, S. M.Roediger, H. L., III (Eds.), Integrating cognitive science with innovative teaching in STEM disciplines (E-reader version, pp. 147–166). St. Louis, MO: Washington University. Google Scholar
• Moon, A., Stanford, C., Cole, R., & Towns, M. ( 2016 ). The nature of students’ chemical reasoning employed in scientific argumentation in physical chemistry . Chemistry Education Research and Practice , 17 (2), 353–364. Google Scholar
• Newman, P. M., & DeCaro, M. S. ( 2019 ). Learning by exploring: How much guidance is optimal? Learning and Instruction , 62 , 49–63. Google Scholar
• NGSS Lead States . ( 2013 ). Next Generation Science Standards: For states, by states . Washington, DC: The National Academies Press Google Scholar
• Nievelstein, F., Van Gog, T., Van Dijck, G., & Boshuizen, H. P. ( 2013 ). The worked example and expertise reversal effect in less structured tasks: Learning to reason about legal cases . Contemporary Educational Psychology , 38 (2), 118–125. Google Scholar
• Polya, G. ( 1957 ). How to solve it (2nd ed.). Princeton, NJ: Princeton University Press. Google Scholar
• Polya, G. ( 1962 Mathematical discovery: On understanding, learning, and teaching problem solving . New York, NY: John Wiley. Google Scholar
• Preszler, R. W. ( 2009 ). Replacing lecture with peer-led workshops improves student learning . CBE—Life Sciences Education , 8 (3), 182–192. Link ,  Google Scholar
• Prince, M. J., & Felder, R. M. ( 2006 ). Inductive teaching and learning methods: Definitions, comparisons, and research bases . Journal of Engineering Education , 95 (2), 123–138. Google Scholar
• Repice, M. D., Sawyer, R. K., Hogrebe, M. C., Brown, P. L., Luesse, S. B., Gealy, D. J., & Frey, R. F. ( 2016 ). Talking through the problems: A study of discourse in peer-led small groups . Chemistry Education Research and Practice , 17 (3), 555–568. Google Scholar
• Rodriguez, J.-M. G., Hunter, K. H., Scharlott, L. J., & Becker, N. M. ( 2020 ). A review of research on process oriented guided inquiry learning: Implications for research and practice . Journal of Chemical Education , 97 (10), 3506–3520. Google Scholar
• Roelle, J., & Berthold, K. ( 2015 ). Effects of comparing contrasting cases on learning from subsequent explanations . Cognition and Instruction , 33 (3), 199–225. Google Scholar
• Schmidt, R. A., & Bjork, R. A. ( 1992 ). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training . Psychological Science , 3 (4), 207–218. Google Scholar
• Shemwell, J. T., Chase, C. C., & Schwartz, D. L. ( 2015 ). Seeking the general explanation: A test of inductive activities for learning and transfer . Journal of Research in Science Teaching , 52 (1), 58–83. Google Scholar
• Simon, H. A., & Newell, A. ( 1972 ). Human problem solving (Vol. 104, No. 9) . Englewood Cliffs, NJ: Prentice-hall. Google Scholar
• Snyder, J. J., Sloane, J. D., Dunk, R. D., & Wiles, J. R. ( 2016 ). Peer-led team learning helps minority students succeed . PLoS Biology , 14 (3), e1002398. Medline ,  Google Scholar
• Sweller, J. ( 1988 ). cognitive load during problem solving: Effects on learning . Cognitive Science , 12 , 257–285. Google Scholar
• Sweller, J., van Merriënboer, J. J., & Paas, F. ( 2019 ). Cognitive architecture and instructional design: 20 years later . Educational Psychology Review , 31 (2), 261–292. Google Scholar
• Toh, P. L. L., & Kapur, M. ( 2017 ). Is having more prerequisite knowledge better for learning from productive failure? Instructional Science , 45 (3), 377–394. Google Scholar
• Tuovinen, J. E., & Sweller, J. ( 1999 ). A comparison of cognitive load associated with discovery learning and worked examples . Journal of Educational Psychology , 91 (2), 334. Google Scholar
• Varma-Nelson, P., Cracolice, M., & Gosser, D. K. ( 2004 ). Peer-led team learning: A student-faculty partnership for transforming the learning environment . In Invention and Impact: Building Excellence in Undergraduate Science, Technology, Engineering, and Mathematics (STEM) Education, Proceedings of Conference Sponsored by the NSF Division of Undergraduate Education and in Collaboration with the AAAS Education and Human Resources Programs, 372. April 16–18, 2004 at Crystal City, VA (pp. 43–48). Washington, DC: American Association for the Advancement of Science. Google Scholar
• Vincent-Ruz, P., Meyer, T., Roe, S. G., & Schunn, C. D. ( 2020 ). Short-term and long-term effects of POGIL in a large-enrollment general chemistry course . Journal of Chemical Education , 97 (5), 1228–1238. Google Scholar
• Wertheimer, M. ( 1959 ). Productive thinking (Rev. ed.). Chicago, IL: University of Chicago Press. Google Scholar
• Wiggins, G., & McTighe, J. ( 1998 ). What is backward design . Understanding by Design , 1 , 7–19. Google Scholar
• Wilson, S. B., & Varma-Nelson, P. ( 2016 ). Small groups, significant impact: A review of peer-led team learning research with implications for STEM education researchers and faculty . Journal of Chemical Education , 93 (10), 1686–1702. Google Scholar
• Zepeda, C. D., Martin, R. S., & Butler, A. C. ( 2020 ). Motivational strategies to engage learners in desirable difficulties . Journal of Applied Research in Memory and Cognition , 9 (4), 468–474. Google Scholar

Submitted: 23 February 2022 Revised: 25 March 2022 Accepted: 30 March 2022

© 2022 R. F. Frey et al. CBE—Life Sciences Education © 2022 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 04 June 2018

Collaborative problem-solving education for the twenty-first-century workforce

• Stephen M. Fiore 1 ,
• Arthur Graesser 2 &
• Samuel Greiff 3  

Nature Human Behaviour volume  2 ,  pages 367–369 ( 2018 ) Cite this article

1629 Accesses

59 Citations

30 Altmetric

Metrics details

The complex research, policy and industrial challenges of the twenty-first century require collaborative problem solving. Assessments suggest that, globally, many graduates lack necessary competencies. There is a pressing need, therefore, to improve and expand teaching of collaborative problem solving in our education systems.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Pre-service teachers becoming researchers: the role of professional learning groups in creating a community of inquiry

• Sandris Zeivots
• , John Douglas Buchanan
•  &  Kimberley Pressick-Kilborn

The Australian Educational Researcher Open Access 20 January 2023

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 digital issues and online access to articles

111,21 € per year

only 9,27 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

Fiore, S. M. et al. Collaborative Problem Solving: Considerations for the National Assessment of Educational Progress (National Center for Educational Statistics, United States Department of Education, Washington DC, 2017).

Graesser, A. C. et al. in Assessment and Teaching of 21st Century Skills. Research and Applications (eds Care, E., Griffin, P. & Wilson, M.) Ch. 5 (Springer International Publishing, Cham, 2018); https://doi.org/10.1007/978-3-319-65368-6_5 .

PISA 2015 Results (Volume V): Collaborative Problem Solving (Organization for Economic Cooperation and Development, 2017); https://doi.org/10.1787/9789264285521-en

National Academies of Sciences, Engineering, and Medicine Education for Life and Work: Transferable Knowledge and Skills in the 21st Century (National Academies Press, Washington DC, 2012); https://doi.org/10.17226/13398

National Research Council Enhancing the Effectiveness of Team Science (National Academies Press, Washington DC, 2015); https://doi.org/10.17226/19007

The Royal Society Assessing Experimental Science in 11–18 Education: New Research Directions (Royal Society Press, 2016); https://royalsociety.org/~/media/events/2016/10/education-conference-report-12-october-2016.pdf

Hart Research Associates Falling Short? College Learning and Career Success (Association of American Colleges and Universities, 2015).

Critical Skills Survey (American Management Association, 2012); https://www.amanet.org/uploaded/2012-Critical-Skills-Survey.pdf

National Academies of Sciences, Engineering, and Medicine Building America’s Skilled Technical Workforce (National Academies Press, Washington DC, 2017); https://doi.org/10.17226/23472

Weinberger, C. J. Rev. Econ. Stat. 96 , 849–861 (2014).

Article   Google Scholar  

Download references

Author information

Authors and affiliations.

University of Central Florida, Orlando, FL, USA

Stephen M. Fiore

University of Memphis, Memphis, TN, USA

Arthur Graesser

University of Luxembourg, Luxembourg City, Luxembourg

Samuel Greiff

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Stephen M. Fiore .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Fiore, S.M., Graesser, A. & Greiff, S. Collaborative problem-solving education for the twenty-first-century workforce. Nat Hum Behav 2 , 367–369 (2018). https://doi.org/10.1038/s41562-018-0363-y

Download citation

Published : 04 June 2018

Issue Date : June 2018

DOI : https://doi.org/10.1038/s41562-018-0363-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Improving collaborative problem-solving skills via automated feedback and scaffolding: a quasi-experimental study with cpscoach 2.0.

• Sidney K. D’Mello
• Nicholas Duran
• Angela E. B. Stewart

User Modeling and User-Adapted Interaction (2024)

• John Douglas Buchanan
• Kimberley Pressick-Kilborn

The Australian Educational Researcher (2023)

Exploring the effects of role scripts and goal-orientation scripts in collaborative problem-solving learning

Education and Information Technologies (2023)

Integrating a collaboration script and group awareness to support group regulation and emotions towards collaborative problem solving

• Matias Rojas
• Miguel Nussbaum
• Danilo Alvares

International Journal of Computer-Supported Collaborative Learning (2022)

Multimodal modeling of collaborative problem-solving facets in triads

• Zachary Keirn

User Modeling and User-Adapted Interaction (2021)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Teaching problem solving: Let students get ‘stuck’ and ‘unstuck’

Subscribe to the center for universal education bulletin, kate mills and km kate mills literacy interventionist - red bank primary school helyn kim helyn kim former brookings expert @helyn_kim.

October 31, 2017

This is the second in a six-part  blog series  on  teaching 21st century skills , including  problem solving ,  metacognition , critical thinking , and collaboration , in classrooms.

In the real world, students encounter problems that are complex, not well defined, and lack a clear solution and approach. They need to be able to identify and apply different strategies to solve these problems. However, problem solving skills do not necessarily develop naturally; they need to be explicitly taught in a way that can be transferred across multiple settings and contexts.

Here’s what Kate Mills, who taught 4 th grade for 10 years at Knollwood School in New Jersey and is now a Literacy Interventionist at Red Bank Primary School, has to say about creating a classroom culture of problem solvers:

Helping my students grow to be people who will be successful outside of the classroom is equally as important as teaching the curriculum. From the first day of school, I intentionally choose language and activities that help to create a classroom culture of problem solvers. I want to produce students who are able to think about achieving a particular goal and manage their mental processes . This is known as metacognition , and research shows that metacognitive skills help students become better problem solvers.

I begin by “normalizing trouble” in the classroom. Peter H. Johnston teaches the importance of normalizing struggle , of naming it, acknowledging it, and calling it what it is: a sign that we’re growing. The goal is for the students to accept challenge and failure as a chance to grow and do better.

I look for every chance to share problems and highlight how the students— not the teachers— worked through those problems. There is, of course, coaching along the way. For example, a science class that is arguing over whose turn it is to build a vehicle will most likely need a teacher to help them find a way to the balance the work in an equitable way. Afterwards, I make it a point to turn it back to the class and say, “Do you see how you …” By naming what it is they did to solve the problem , students can be more independent and productive as they apply and adapt their thinking when engaging in future complex tasks.

After a few weeks, most of the class understands that the teachers aren’t there to solve problems for the students, but to support them in solving the problems themselves. With that important part of our classroom culture established, we can move to focusing on the strategies that students might need.

Here’s one way I do this in the classroom:

I show the broken escalator video to the class. Since my students are fourth graders, they think it’s hilarious and immediately start exclaiming, “Just get off! Walk!”

When the video is over, I say, “Many of us, probably all of us, are like the man in the video yelling for help when we get stuck. When we get stuck, we stop and immediately say ‘Help!’ instead of embracing the challenge and trying new ways to work through it.” I often introduce this lesson during math class, but it can apply to any area of our lives, and I can refer to the experience and conversation we had during any part of our day.

Research shows that just because students know the strategies does not mean they will engage in the appropriate strategies. Therefore, I try to provide opportunities where students can explicitly practice learning how, when, and why to use which strategies effectively  so that they can become self-directed learners.

For example, I give students a math problem that will make many of them feel “stuck”. I will say, “Your job is to get yourselves stuck—or to allow yourselves to get stuck on this problem—and then work through it, being mindful of how you’re getting yourselves unstuck.” As students work, I check-in to help them name their process: “How did you get yourself unstuck?” or “What was your first step? What are you doing now? What might you try next?” As students talk about their process, I’ll add to a list of strategies that students are using and, if they are struggling, help students name a specific process. For instance, if a student says he wrote the information from the math problem down and points to a chart, I will say: “Oh that’s interesting. You pulled the important information from the problem out and organized it into a chart.” In this way, I am giving him the language to match what he did, so that he now has a strategy he could use in other times of struggle.

The charts grow with us over time and are something that we refer to when students are stuck or struggling. They become a resource for students and a way for them to talk about their process when they are reflecting on and monitoring what did or did not work.

For me, as a teacher, it is important that I create a classroom environment in which students are problem solvers. This helps tie struggles to strategies so that the students will not only see value in working harder but in working smarter by trying new and different strategies and revising their process. In doing so, they will more successful the next time around.

Related Content

Esther Care, Helyn Kim, Alvin Vista

October 17, 2017

David Owen, Alvin Vista

November 15, 2017

Loren Clarke, Esther Care

December 5, 2017

Global Education K-12 Education

Global Economy and Development

Center for Universal Education

Brad Olsen, John McIntosh

April 3, 2024

Darcy Hutchins, Emily Markovich Morris, Laura Nora, Carolina Campos, Adelaida Gómez Vergara, Nancy G. Gordon, Esmeralda Macana, Karen Robertson

March 28, 2024

Jennifer B. Ayscue, Kfir Mordechay, David Mickey-Pabello

March 26, 2024

• Author Rights
• Diversity, Equity & Inclusion

• JOLE 2023 Special Issue
• Editorial Staff
• 20th Anniversary Issue
• The Development of Problem-Solving Skills for Aspiring Educational Leaders

Jeremy D. Visone 10.12806/V17/I4/R3

Introduction

Solving problems is a quintessential aspect of the role of an educational leader. In particular, building leaders, such as principals, assistant principals, and deans of students, are frequently beset by situations that are complex, unique, and open-ended. There are often many possible pathways to resolve the situations, and an astute educational leader needs to consider many factors and constituencies before determining a plan of action. The realm of problem solving might include student misconduct, personnel matters, parental complaints, school culture, instructional leadership, as well as many other aspects of educational administration. Much consideration has been given to the development of problem-solving skills for educational leaders. This study was designed to answer the following research question: “How do aspiring educational leaders’ problem solving skills, as well as perceptions of their problem-solving skills, develop during a year-long graduate course sequence focused on school-level leadership that includes the presentation of real-world scenarios?” This mixed-methods study extends research about the development of problem-solving skills conducted with acting administrators (Leithwood & Steinbach, 1992, 1995).

The Nature of Problems

Before examining how educational leaders can process and solve problems effectively, it is worth considering the nature of problems. Allison (1996) posited simply that problems are situations that require thought and/or actions. Further, there are different types of problems presented to educational leaders. First, there are  well-structured problems , which can be defined as those with clear goals and relatively prescribed resolution pathways, including an easy way of determining whether goals were met (Allison, 1996).

Conversely,  ill-structured problems  are those with more open-ended profiles, whereby the goals, resolution pathways, or evidence of success are not necessarily clear. These types of problems could also be considered  unstructured  (Leithwood & Steinbach, 1995) or  open-design  (Allison, 1996). Many of the problems presented to educational leaders are unstructured problems. For example, a principal must decide how to discipline children who misbehave, taking into consideration their disciplinary history, rules and protocols of the school, and other contextual factors; determine how best to raise student achievement (Duke, 2014); and resolve personnel disputes among staff members. None of these problems point to singular solutions that can be identified as “right” or “wrong.” Surely there are responses that are less desirable than others (i.e. suspension or recommendation for expulsion for minor infractions), but, with justification and context, many possible solutions exist.

Problem-Solving Perspectives and Models

Various authors have shared perspectives about effective problem solving. Marzano, Waters, and McNulty (2005) outlined the “21 Responsibilities of the School Leader.” These responsibilities are highly correlated with student achievement based upon the authors’ meta- analysis of 69 studies about leadership’s effect on student achievement. The most highly correlated of the responsibilities was  situational awareness , which refers to understanding the school deeply enough to anticipate what might go wrong from day-to-day, navigate the individuals and groups within the school, and recognize issues that might surface at a later time (Marzano et al., 2005). Though the authors discuss the utility of situational awareness for long- term, large-scale decision making, in order for an educational leader to effectively solve the daily problems that come her way, she must again have a sense of situational awareness, lest she make seemingly smaller-scale decisions that will lead to large-scale problems later.

Other authors have focused on problems that can be considered more aligned with the daily work of educational leaders. Considering the problem-type classification dichotomies of Allison (1996) and Leithwood and Steinbach (1995), problems that educational leaders face on a daily basis can be identified as either well-structured or unstructured. Various authors have developed problem-solving models focused on unstructured problems (Bolman & Deal, 2008; Leithwood & Steinbach, 1995; Simon, 1993), and these models will be explored next.

Simon (1993) outlined three phases of the decision-making process. The first is to find problems that need attention. Though many problems of educational leaders are presented directly to them via, for example, an adult referring a child for discipline, a parent registering a complaint about a staff member, or a staff member describing a grievance with a colleague, there is a corollary skill of identifying what problems—of the many that come across one’s desk— require immediate attention, or ultimately, any attention, at all. Second, Simon identified “designing possible courses of action” (p. 395). Finally, educational leaders must evaluate the quality of their decisions. From this point of having selected a viable and positively evaluated potential solution pathway, implementation takes place.

Bolman and Deal (2008) outlined a model of reframing problems using four different frames, through which problems of practice can be viewed. These frames provide leaders with a more complete set of perspectives than they would likely utilize on their own. The  structural frame  represents the procedural and systems-oriented aspects of an organization. Within this frame, a leader might ask whether there is a supervisory relationship involved in a problem, if a protocol exists to solve such a problem, or what efficiencies or logical processes can help steer a leader toward a resolution that meets organizational goals. The  human resource frame  refers to the needs of individuals within the organization. A leader might try to solve a problem of practice with the needs of constituents in mind, considering the development of employees and the balance between their satisfaction and intellectual stimulation and the organization’s needs. The  political frame  includes the often competing interests among individuals and groups within the organization, whereby alliances and negotiations are needed to navigate the potential minefield of many groups’ overlapping aims. From the political frame, a leader could consider what the interpersonal costs will be for the leader and organization among different constituent groups, based upon which alternatives are selected. Last, the  symbolic frame  includes elements of meaning within an organization, such as traditions, unspoken rules, and myths. A leader may need to consider this frame when proposing a solution that might interfere with a long-standing organizational tradition.

Bolman and Deal (2008) identified the political and symbolic frames as weaknesses in most leaders’ consideration of problems of practice, and the weakness in recognizing political aspects of decision making for educational leaders was corroborated by Johnson and Kruse (2009). An implication for leadership preparation is to instruct students in the considerations of these frames and promote their utility when examining problems.

Authors have noted that experts use different processes than novice problem solvers (Simon, 1993; VanLehn, 1991). An application of this would be Simon’s (1993) assertion that experts can rely on their extensive experience to remember solutions to many problems, without having to rely on an extensive analytical process. Further, they may not even consider a “problem” identified by a novice a problem, at all. With respect to educational leaders, Leithwood and Steinbach (1992, 1995) outlined a set of competencies possessed by expert principals, when compared to their typical counterparts. Expert principals were better at identifying the nature of problems; possessing a sense of priority, difficulty, how to proceed, and connectedness to prior situations; setting meaningful goals for problem solving, such as seeking goals that are student-centered and knowledge-focused; using guiding principles and long-term purposes when determining the best courses of action; seeing fewer obstacles and constraints when presented with problems; outlining detailed plans for action that include gathering extensive information to inform decisions along the plan’s pathway; and responding with confidence and calm to problem solving. Next, I will examine how problem-solving skills are developed.

Preparation for Educational Leadership Problem Solving

How can the preparation of leaders move candidates toward the competencies of expert principals? After all, leading a school has been shown to be a remarkably complex enterprise (Hallinger & McCary, 1990; Leithwood & Steinbach, 1992), especially if the school is one where student achievement is below expectations (Duke, 2014), and the framing of problems by educational leaders has been espoused as a critically important enterprise (Bolman & Deal, 2008; Dimmock, 1996; Johnson & Kruse, 2009; Leithwood & Steinbach, 1992, 1995; Myran & Sutherland, 2016). In other disciplines, such as business management, simulations and case studies are used to foster problem-solving skills for aspiring leaders (Rochford & Borchert, 2011; Salas, Wildman, & Piccolo, 2009), and attention to problem-solving skills has been identified as an essential curricular component in the training of journalism and mass communication students (Bronstein & Fitzpatrick, 2015). Could such real-world problem solving methodologies be effective in the preparation of educational leaders? In a seminal study about problem solving for educational leaders, Leithwood and Steinbach (1992, 1995) sought to determine if effective problem-solving expertise could be explicitly taught, and, if so, could teaching problem- processing expertise be helpful in moving novices toward expert competence? Over the course of four months and four separate learning sessions, participants in the control group were explicitly taught subskills within six problem-solving components: interpretation of the problem for priority, perceived difficulty, data needed for further action, and anecdotes of prior experience that can inform action; goals for solving the problem; large-scale principles that guide decision making; barriers or obstacles that need to be overcome; possible courses of action; and the confidence of the leader to solve the problem. The authors asserted that providing conditions to participants that included models of effective problem-solving, feedback, increasingly complex problem-solving demands, frequent opportunities for practice, group problem-solving, individual reflection, authentic problems, and help to stimulate metacognition and reflection would result in educational leaders improving their problem-solving skills.

The authors used two experts’ ratings of participants’ problem-solving for both process (their methods of attacking the problem) and product (their solutions) using a 0-3 scale in a pretest-posttest design. They found significant increases in some problem-solving skills (problem interpretation, goal setting, and identification of barriers or obstacles that need to be overcome) after explicit instruction (Leithwood & Steinbach, 1992, 1995). They recommended conducting more research on the preparation of educational leaders, with particular respect to approaches that would improve the aspiring leaders’ problem-solving skills.

Solving problems for practicing principals could be described as constructivist, since most principals do solve problems within a social context of other stakeholders, such as teachers, parents, and students (Leithwood & Steinbach, 1992). Thus, some authors have examined providing opportunities for novice or aspiring leaders to construct meaning from novel scenarios using the benefits of, for example, others’ point of view, expert modeling, simulations, and prior knowledge (Duke, 2014; Leithwood & Steinbach, 1992, 1995; Myran & Sutherland, 2016; Shapira-Lishchinsky, 2015). Such collaborative inquiry has been effective for teachers, as well (DeLuca, Bolden, & Chan, 2017). Such learning can be considered consistent with the ideas of other social constructivist theorists (Berger & Luckmann, 1966; Vygotsky, 1978) as well, since individuals are working together to construct meaning, and they are pushing into areas of uncertainty and lack of expertise.

Shapira-Lishchinsky (2015) added some intriguing findings and recommendations to those of Leithwood and Steinbach (1992, 1995). In this study, 50 teachers with various leadership roles in their schools were presented regularly with ethical dilemmas during their coursework. Participants either interacted with the dilemmas as members of a role play or by observing those chosen. When the role play was completed, the entire group debriefed and discussed the ethical dilemmas and role-playing participants’ treatment of the issues. This method was shown, through qualitative analysis of participants’ discussions during the simulations, to produce rich dialogue and allow for a safe and controlled treatment of difficult issues. As such, the use of simulations was presented as a viable means through which to prepare aspiring educational leaders. Further, the author suggested the use of further studies with simulation-based learning that seek to gain information about aspiring leaders’ self-efficacy and psychological empowerment. A notable example of project-based scenarios in a virtual collaboration environment to prepare educational leaders is the work of Howard, McClannon, and Wallace (2014). Shapira-Lishchinsky (2015) also recommended similar research in other developed countries to observe the utility of the approaches of simulation and social constructivism to examine them for a wider and diverse aspiring administrator candidate pool.

Further, in an extensive review of prior research studies on the subject, Hallinger and Bridges (2017) noted that Problem-Based Learning (PBL), though applied successfully in other professions and written about extensively (Hallinger & Bridges, 1993, 2017; Stentoft, 2017), was relatively unheralded in the preparation of educational leaders. According to the authors, characteristics of PBL included problems replacing theory as the organization of course content, student-led group work, creation of simulated products by students, increased student ownership over learning, and feedback along the way from professors. Their review noted that PBL had positive aspects for participants, such as increased motivation, real-world connections, and positive pressure that resulted from working with a team. However, participants also expressed concerns about time constraints, lack of structure, and interpersonal dynamics within their teams. There were positive effects found on aspiring leaders’ problem-solving skill development with PBL (Copland, 2000; Hallinger & Bridges, 2017). Though PBL is much more prescribed than the scenarios strategy described in the Methods section below, the applicability of real-world problems to the preparation of educational leaders is summarized well by Copland (2000):

[I]nstructional practices that activate prior knowledge and situate learning in contexts similar to those encountered in practice are associated with the development of students’ ability to understand and frame problems. Moreover, the incorporation of debriefing techniques that encourage students’ elaboration of knowledge and reflection on learning appear to help students solidify a way of thinking about problems. (p. 604)

This study involved a one-group pretest-posttest design. No control group was assigned, as the pedagogical strategy in question—the use of real-world scenarios to build problem-solving skill for aspiring educational leaders—is integral to the school’s curriculum that prepares leaders, and, therefore, it is unethical to deny to student participants (Gay & Airasian, 2003). Thus, all participants were provided instruction with the use of real-world scenarios.

Participants.  Graduate students at a regional, comprehensive public university in the Northeast obtaining a 6 th -year degree (equivalent to a second master’s degree) in educational leadership and preparing for certification as educational administrators served as participants. Specifically, students in three sections of the same full-year, two-course sequence, entitled “School Leadership I and II” were invited to participate. This particular course was selected from the degree course sequence, as it deals most directly with the problem-solving nature and daily work of school administrators. Some key outcomes of the course include students using data to drive school improvement action plans, communicating effectively with a variety of stakeholders, creating a safe and caring school climate, creating and maintaining a strategic and viable school budget, articulating all the steps in a hiring process for teachers and administrators, and leading with cultural proficiency.

The three sections were taught by two different professors. The professors used real- world scenarios in at least half of their class meetings throughout the year, or in approximately 15 classes throughout the year. During these classes, students were presented with realistic situations that have occurred, or could occur, in actual public schools. Students worked with their classmates to determine potential solutions to the problems and then discussed their responses as a whole class under the direction of their professor, a master practitioner. Both professors were active school administrators, with more than 25 years combined educational leadership experience in public schools. It should be noted that the scenario presentation and discussions took place during the class sessions, only. These were not presented for homework or in online forums.

Of the 44 students in these three sections, 37 volunteered to participate at some point in the data collection sequence, but not all students in the pretest session attended the posttest session months later and vice versa. As a result, only 20 students’ data were used for the matched pairs analysis. All 37 participants were certified professional educators in public schools in Connecticut. The participants’ professional roles varied and included classroom teachers, instructional coaches, related service personnel, unified arts teachers, as well as other non- administrative educational roles. Characteristics of participants in the overall and matched pairs groups can be found in Table 1.

Table 1 Participant Characteristics

Procedure.  Participants’ data were compared between a fall of 2016 baseline data collection period and a spring of 2017 posttest data collection period. During the fall data collection period, participants were randomly assigned one of two versions of a Google Forms survey. After items about participant characteristics, the survey consisted of 11 items designed to elicit quantitative and qualitative data about participants’ perceptions of their problem-solving abilities, as well as their ability to address real-world problems faced by educational leaders. The participants were asked to rate their perception of their situational awareness, flexibility, and problem solving ability on a 10-point (1-10) Likert scale, following operational definitions of the terms (Marzano, Waters, & McNulty, 2005; Winter, 1982). They were asked, for each construct, to write open-ended responses to justify their numerical rating. They were then asked to write what they perceived they still needed to improve their problem-solving skills. The final four items included two real-world, unstructured, problem-based scenarios for which participants were asked to create plans of action. They were also asked to rate their problem-solving confidence with respect to their proposed action plans for each scenario on a 4-point (0-3) Likert scale.

During the spring data collection period, participants accessed the opposite version of the Google Forms survey from the one they completed in the fall. All items were identical on the two survey versions, except the scenarios, which were different on each survey version. The use of two versions was to ensure that any differences in perceived or actual difficulty among the four scenarios provided would not alter results based upon the timing of participant access (Leithwood & Steinbach, 1995). In order to link participants’ fall and spring data in a confidential manner, participants created a unique, six-digit alphanumeric code.

A focus group interview followed each spring data collection session. The interviews were recorded to allow for accurate transcription. The list of standard interview questions can be found in Table 2. This interview protocol was designed to elicit qualitative data with respect to aspiring educational leaders’ perceptions about their developing problem-solving abilities.

Table 2 Focus Group Interview Questions ___________________________________________________________________________________________

Please describe the development of your problem-solving skills as an aspiring educational leader over the course of this school year. In what ways have you improved your skills? Be as specific as you can.

What has been helpful to you (i.e. coursework, readings, experiences, etc.) in this development of your problem-solving skills? Why?

What do you believe you still need for the development in your problem-solving skills as an aspiring educational leader?

Discuss your perception of your ability to problem solve as an aspiring educational leader. How has this changed from the beginning of this school year? Why?

Please add anything else you perceive is relevant to this conversation about the development of your problem-solving skills as an aspiring educational leader.

___________________________________________________________________________________________

Data Analysis.

Quantitative data .  Data were obtained from participants’ responses to Likert-scale items relating to their confidence levels with respect to aspects of problem solving, as well as from the rating of participants’ responses to the given scenarios  against a rubric. The educational leadership problem-solving rubric chosen (Leithwood & Steinbach, 1995) was used with permission, and it reflects the authors’ work with explicitly teaching practicing educational leaders components of problem solving. The adapted rubric can be found in Figure 1. Through the use of this rubric, each individual response by a participant to a presented scenario was assigned a score from 0-15. It should be noted that affect data (representing the final 3 possible points on the 18-point rubric) were obtained via participants’ self-reporting their confidence with respect to their proposed plans of action. To align with the rubric, participants self-assessed their confidence through this item with a 0-3 scale.

0 = No Use of the Subskill 1 = There is Some Indication of Use of the Subskill 2 = The Subskill is Present to Some Degree 3 = The Subskill is Present to a Marked Degree; This is a Fine Example of this Subskill

Figure 1.  Problem-solving model for unstructured problems. Adapted from “Expert Problem Solving: Evidence from School and District Leaders,” by K. Leithwood and R. Steinbach, pp. 284-285. Copyright 1995 by the State University of New York Press.

I compared Likert-scale items and rubric scores via descriptive statistics and rubric scores also via a paired sample  t -test and Cohen’s  d , all using the software program IBM SPSS. I did not compare the Likert-scale items about situational awareness, flexibility, and problem solving ability with  t -tests or Cohen’s  d , since these items did not represent a validated instrument. They were only single items based upon participants’ ratings compared to literature-based definitions. However, the value of the comparison of means from fall to spring was triangulated with qualitative results to provide meaning. For example, to say that participants’ self-assessment ratings for perceived problem-solving abilities increased, I examined both the mean difference for items from fall to spring and what participants shared throughout the qualitative survey items and focus group interviews.

Prior to scoring participants’ responses to the scenarios using the rubric, and in an effort to maximize the content validity of the rubric scores, I calibrated my use of the rubric with two experts from the field. Two celebrated principals, representing more than 45 combined years of experience in school-level administration, collaboratively and comparatively scored participant responses. Prior to scoring, the team worked collaboratively to construct appropriate and comprehensive exemplar responses to the four problem-solving scenarios. Then the team blindly scored fall pretest scenario responses using the Leithwood and Steinbach (1995) rubric, and upon comparing scores, the interrater reliability correlation coefficient was .941, indicating a high degree of agreement throughout the team.

Qualitative data.  These data were obtained from open-ended items on the survey, including participants’ responses to the given scenarios, as well as the focus group interview transcripts. I analyzed qualitative data consistent with the grounded theory principles of Strauss and Corbin (1998) and the constant comparative methods of Glaser (1965), including a period of open coding of results, leading to axial coding to determine the codes’ dimensions and relationships between categories and their subcategories, and selective coding to arrive at themes. Throughout the entire data analysis process, I repeatedly returned to raw data to determine the applicability of emergent codes to previously analyzed data. Some categorical codes based upon the review of literature were included in the initial coding process. These codes were derived from the existing theoretical problem-solving models of Bolman and Deal (2008) and Leithwood and Steinbach (1995). These codes included  modeling ,  relationships , and  best for kids . Open codes that emerged from the participants’ responses included  experience ,  personality traits ,  current job/role , and  team . Axial coding revealed, for example, that current jobs or roles cited, intuitively, provided both sufficient building-wide perspective and situational memory (i.e. for special education teachers and school counselors) and insufficient experiences (i.e. for classroom teachers) to solve the given problems with confidence. From such understandings of the codes, categories, and their dimensions, themes were developed.

Quantitative Results.   First, participants’ overall, aggregate responses (not matched pairs) were compared from the fall to spring, descriptively. These findings are outlined in Table  3. As is seen in the table, each item saw a modest increase over the course of the year. Participant perceptions of their problem-solving abilities across the three constructs presented (situational awareness, flexibility, and problem solving) did increase over the course of the year, as did the average group score for the problem-solving scenarios. However, due to participant differences in the two data collection periods, these aggregate averages do not represent a matched-pair dataset.

Table 3 Fall to Spring Comparison of Likert-Scale and Rubric-Scored Items

a  These problem-solving dimensions from literature were rated by participants on a scale from 1- 10. b  Participants received a rubric score for each scenario between 0-18. Participants’ two scenario scores for each data collection period (fall, spring) were averaged to arrive at the scores represented here.

In order to determine the statistical significance of the increase in participants’ problem- solving rubric scores, a paired-samples  t -test was applied to the fall ( M  = 9.15;  SD  = 2.1) and spring ( M  = 9.25;  SD  = 2.3) averages. Recall that 20 participants had valid surveys for both the fall and spring. The  t -test ( t  = -.153;  df  = 19;  p  = .880) revealed no statistically significant change from fall to spring, despite the minor increase (0.10). I applied Cohen’s  d  to calculate the effect size. The small sample size ( n  = 20) for the paired-sample  t -test may have contributed to the lack of statistical significance. However, standard deviations were also relatively small, so the question of effect size was of particular importance. Cohen’s  d  was 0.05, which is also very small, indicating that little change—really no improvement, from a statistical standpoint—in participants’ ability to create viable action plans to solve real-world problems occurred throughout the year. However, the participants’ perceptions of their problem-solving abilities did increase, as evidenced by the increases in the paired-samples perception means shown in Table 3, though these data were only examined descriptively (from a quantitative perspective) due to the fact that these questions were individual items that are not part of a validated instrument.

Qualitative Results.   Participant responses to open-ended items on the questionnaire, responses to the scenarios, and oral responses to focus group interview questions served as sources of qualitative data. Since the responses to the scenarios were focused on participant competence with problem solving, as measured by the aforementioned rubric (Leithwood &  Steinbach, 1995), these data were examined separately from data collected from the other two sources.

Responses to scenarios.  As noted, participants’ rubric ratings for the scenarios did not display a statistically significant increase from fall to spring. As such, this outline will not focus upon changes in responses from fall to spring. Rather, I examined the responses, overall, through the lens of the Leithwood and Steinbach (1995) problem-solving framework indicators against which they were rated. Participants typically had outlined reasonable, appropriate, and logical solution processes. For example, in a potential bullying case scenario, two different participants offered, “I would speak to the other [students] individually if they have said or done anything mean to other student [ sic ] and be clear that it is not tolerable and will result in major consequences” and “I would initiate an investigation into the situation beginning with [an] interview with the four girls.” These responses reflect actions that the consulted experts anticipated from participants and deemed as logical and needed interventions. However, these two participants omitted other needed steps, such as addressing the bullied student’s mental health needs, based upon her mother’s report of suicidal ideations. Accordingly, participants earned points for reasonable and logical responses very consistently, yet, few full-credit responses were observed.

Problem interpretation scores were much more varied. For this indicator, some participants were able to identify many, if not all, the major issues in the scenarios that needed attention. For example, for a scenario where two teachers were not interacting professionally toward each other, many participants correctly identified that this particular scenario could include elements of sexual harassment, professionalism, teaching competence, and personality conflict. However, many other participants missed at least two of these key elements of the problem, leaving their solution processes incomplete. The categories of (a) goals and (b) principles and values also displayed a similarly wide distribution of response ratings.

One category, constraints, presented consistent difficulty for the participants. Ratings were routinely 0 and 1. Participants could not consistently report what barriers or obstacles would need addressing prior to success with their proposed solutions. To be clear, it was not a matter of participants listing invalid or unrealistic barriers or obstacles; rather, the participants were typically omitting constraints altogether from their responses. For example, for a scenario involving staff members arriving late and unprepared to data team meetings, many participants did not identify that a school culture of not valuing data-driven decision making or lack of norms for data team work could be constraints that the principal could likely face prior to reaching a successful resolution.

Responses to open-ended items.  When asked for rationale regarding their ratings for situational awareness, flexibility, and problem solving, participants provided open-ended responses. These responses revealed patterns worth considering, and, again, this discussion will consider, in aggregate, responses made in both the pre- and post- data collection periods, again due to the similarities in responses between the two data collection periods. The most frequently observed code (112 incidences) was  experience . Closely related were the codes  current job/role  (50 incidences). Together, these codes typically represented a theme that participants were linking their confidence with respect to problem solving with their exposure (or lack thereof) in their professional work. For example, a participant reported, “As a school counselor, I have a lot of contact with many stakeholders in the school -admin [ sic ], parents, teachers, staff, etc. I feel that I have a pretty good handle on the systemic issues.” This example is one of many where individuals working in counseling, instructional coaching, special education, and other support roles expressed their advanced levels of perspective based upon their regular contact with many stakeholders, including administrators. Thus, they felt they had more prior knowledge and situational memory about problems in their schools.

However, this category of codes also included those, mostly classroom or unified arts teachers, who expressed that their relative lack of experiences outside their own classrooms limited their perspective for larger-scale problem solving. One teacher succinctly summarized this sentiment, “I have limited experience in being part of situations outside of my classroom.” Another focused on the general problem solving skill in her classroom not necessarily translating to confidence with problem solving at the school level: “I feel that I have a high situational awareness as a teacher in the classroom, but as I move through these leadership programs I find that I struggle to take the perspective of a leader.” These experiences were presented in opposition to their book learning or university training. There were a number of instances (65 combined) of references to the value of readings, class discussions, group work, scenarios presented, research, and coursework in the spring survey. When asked what the participants need more, again, experience was referenced often. One participant summarized this concept, “I think that I, personally, need more experience in the day-to-day . . . setting.” Another specifically separated experiences from scenario work, “[T]here is [ sic ] some things you can not [ sic ] learn from merely discussing a ‘what if” scenario. A seasoned administrator learns problem solving skills on the job.”

Another frequently cited code was  personality traits  (63 incidences), which involved participants linking elements of their own personalities to their perceived abilities to process problems, almost exclusively from an assets perspective. Examples of traits identified by participants as potentially helpful in problem solving included: open-mindedness, affinity for working with others, not being judgmental, approachability, listening skills, and flexibility. One teacher exemplified this general approach by indicating, “I feel that I am a good listener in regards to inviting opinions. I enjoy learning through cooperation and am always willing to adapt my teaching to fit needs of the learners.” However, rare statements of personality traits interfering with problem solving included, “I find it hard to trust others [ sic ] abilities” and “my personal thoughts and biases.”

Another important category of the participant responses involved connections with others. First, there were many references to  relationships  (27 incidences), mostly from the perspective that building positive relationships leads to greater problem-solving ability, as the aspiring leader knows stakeholders better and can rely on them due to the history of positive interactions. One participant framed this idea from a deficit perspective, “Not knowing all the outlying relationships among staff members makes situational awareness difficult.” Another identified that established positive relationships are already helpful to an aspiring leader, “I have strong rapport with fellow staff members and administrators in my building.” In a related way, many instances of the code  team  were identified (29). These references overwhelmingly identified that solving problems within a team context is helpful. One participant stated, “I often team with people to discuss possible solutions,” while another elaborated,

I recognize that sometimes problems may arise for which I am not the most qualified or may not have the best answer. I realize that I may need to rely on others or seek out help/opinions to ensure that I make the appropriate decision.

Overall, participants recognized that problem-solving for leaders does not typically occur in a vacuum.

Responses to focus group interview questions.  As with the open-ended responses, patterns were evident in the interview responses, and many of these findings were supportive of the aforementioned themes. First, participants frequently referenced the power of group work to help build their understanding about problems and possible solutions. One participant stated, “hearing other people talk and realizing other concerns that you may not have thought of . . . even as a teacher sometimes, you look at it this way, and someone else says to see it this way.” Another added, “seeing it from a variety of persons [ sic ] point of views. How one person was looking at it, and how another person was looking at it was really helpful.” Also, the participants noted the quality of the discussion was a direct result of “professors who have had real-life experience” as practicing educational leaders, so they could add more realistic feedback and insight to the discussions.

Perhaps most notable in the participant responses during the focus groups was the emphasis on the value of real-world scenarios for the students. These were referenced, without prompting, in all three focus groups by many participants. Answers to the question about what has been most helpful in the development of their problem-solving skills included, “I think the real-world application we are doing,” “I think being presented with all the scenarios,” and “[the professor] brought a lot of real situations.”

With respect to what participants believed they still needed to become better and more confident problem solvers, two patterns emerged. First, students recognized that they have much more to learn, especially with respect to policy and law. It is noteworthy that, with few exceptions, these students had not taken the policy or law courses in the program, and they had not yet completed their administrative internships. Some students actually reported rating themselves as less capable problem solvers in the spring because they now understood more clearly what they lacked in knowledge. One student exemplified this sentiment, “I might have graded myself higher in the fall than I did now . . . [I now can] self identify areas I could improve in that I was not as aware of.” Less confidence in the spring was a minority opinion, however. In a more typical response, another participant stated, “I feel much more prepared for that than I did at the beginning of the year.”

Overall, the most frequently discussed future need identified was experience, either through the administrative internship or work as a formal school administrator. Several students summarized this idea, “That real-world experience to have to deal with it without being able to talk to 8 other people before having to deal with it . . . until you are the person . . . you don’t know” and “They tell you all they want. You don’t know it until you are in it.” Overall, most participants perceived themselves to have grown as problem solvers, but they overwhelmingly recognized that they needed more learning and experience to become confident and effective problem solvers.

This study continues a research pathway about the development of problem-solving skills for administrators by focusing on their preparation. The participants did not see a significant increase in their problem-solving skills over the year-long course in educational leadership.

Whereas, this finding is not consistent with the findings of others who focused on the development of problem-solving skills for school leaders (Leithwood & Steinbach, 1995; Shapira-Lishchinsky, 2015), nor is it consistent with PBL research about the benefits of that approach for aspiring educational leaders (Copland, 2000; Hallinger & Bridges, 2017), it is important to note that the participants in this study were at a different point in their careers. First, they were aspirants, as opposed to practicing leaders. Also, the studied intervention (scenarios) was not the same or nearly as comprehensive as the prescriptive PBL approach. Further, unlike the participants in either the practicing leader or PBL studies, because these individuals had not yet had their internship experiences, they had no practical work as educational leaders. This theme of lacking practical experience was observed in both open-ended responses and focus group interviews, with participants pointing to their upcoming internship experiences, or even their eventual work as administrators, as a key missing piece of their preparation.

Despite the participants’ lack of real gains across the year of preparation in their problem- solving scores, the participants did, generally, report an increase in their confidence in problem solving, which they attributed to a number of factors. The first was the theme of real-world context. This finding was consistent with others who have advocated for teaching problem solving through real-world scenarios (Duke, 2014; Leithwood & Steinbach, 1992, 1995; Myran & Sutherland, 2016; Shapira-Lishchinsky, 2015). This study further adds to this conversation, not only a corroboration of the importance of this method (at least in aspiring leaders’ minds), but also that participants specifically recognized their professors’ experiences as school administrators as important for providing examples, context, and credibility to the work in the classroom.

In addition to the scenario approach, the participants also recognized the importance of learning from one another. In addition to the experiences of their practitioner-professors, many participants espoused the value of hearing the diverse perspectives of other students. The use of peer discussion was also an element of instruction in the referenced studies (Leithwood & Steinbach, 1995; Shapira-Lishchinsky, 2015), corroborating the power of aspiring leaders learning from one another and supporting existing literature about the social nature of problem solving (Berger & Luckmann, 1966; Leithwood & Steinbach, 1992; Vygotsky, 1978).

Finally, the ultimate theme identified through this study is the need for real-world experience in the field as an administrator or intern. It is simply not enough to learn about problem solving or learn the background knowledge needed to solve problems, even when the problems presented are real-world in nature. Scenarios are not enough for aspiring leaders to perceive their problem-solving abilities to be adequate or for their actual problem-solving abilities to improve. They need to be, as some of the participants reasoned, in positions of actual responsibility, where the weight of their decisions will have tangible impacts on stakeholders, including students.

The study of participants’ responses to the scenarios connected to the Four Frames model of Bolman and Deal (2008). The element for which participants received the consistently highest scores was identifying solution processes. This area might most logically be connected to the structural and human resource frames, as solutions typically involve working to meet individuals’ needs, as is necessary in the human resource frame, and attending to protocols and procedures, which is the essence of the structural frame. As identified above, the political and symbolic frames have been cited by the authors as the most underdeveloped by educational leaders, and this assertion is corroborated by the finding in this study that participants struggled the most with identifying constraints, which can sometimes arise from an understanding of the competing personal interests in an organization (political frame) and the underlying meaning behind aspects of an organization (symbolic frame), such as unspoken rules and traditions. The lack of success identifying constraints is also consistent with participants’ statements that they needed actual experiences in leadership roles, during which they would likely encounter, firsthand, the types of constraints they were unable to articulate for the given scenarios. Simply, they had not yet “lived” these types of obstacles.

The study includes several notable limitations. First, the study’s size is limited, particularly with only 20 participants’ data available for the matched pairs analysis. Further, this study was conducted at one university, within one particular certification program, and over three sections of one course, which represented about one-half of the time students spend in the program. It is likely that more gains in problem-solving ability and confidence would have been observed if this study was continued through the internship year. Also, the study did not include a control group. The lack of an experimental design limits the power of conclusions about causality. However, this limitation is mitigated by two factors. First, the results did not indicate a statistically significant improvement, so there is not a need to attribute a gain score to a particular variable (i.e. use of scenarios), anyway, and, second, the qualitative results did reveal the perceived value for participants in the use of scenarios, without any prompting of the researcher. Finally, the participant pool was not particularly diverse, though this fact is not particularly unusual for the selected university, in general, representing a contemporary challenge the university’s state is facing to educate its increasingly diverse student population, with a teaching and administrative workforce that is predominantly White.

The findings in this study invite further research. In addressing some of the limitations identified here, expanding this study to include aspiring administrators across other institutions representing different areas of the United States and other developed countries, would provide a more generalizable set of results. Further, studying the development of problem-solving skills during the administrative internship experience would also add to the work outlined here by considering the practical experience of participants.

In short, this study illustrates for those who prepare educational leaders the value of using scenarios in increasing aspiring leaders’ confidence and knowledge. However, intuitively, scenarios alone are not enough to engender significant change in their actual problem-solving abilities. Whereas, real-world context is important to the development of aspiring educational leaders’ problem-solving skills, the best context is likely to be the real work of administration.

Allison, D. J. (1996). Problem finding, classification and interpretation: In search of a theory of administrative problem processing. In K. Leithwood, J. Chapman, D. Corson, P. Hallinger, & A. Hart (Eds.),  International handbook of educational leadership and administration  (pp. 477–549). Norwell, MA: Kluwer Academic.

Berger, P. L., & Luckmann, T. (1966).  The social construction of reality . Garden City, NJ: Doubleday.

Bolman, L. G., & Deal, T. E. (2008).  Re-framing organizations: Artistry, choice and leadership  (4th ed.). San Francisco: Jossey Bass.

Bronstein, C., & Fitzpatrick, K. R. (2015). Preparing tomorrow’s leaders: Integrating leadership development in journalism and mass communication education.  Journalism & Mass Communication Educator, 70 (1), 75–88. https://doi.org/10.1177/1077695814566199

Copland, M. A. (2000). Problem-based learning and prospective principals’ problem-framing ability.  Educational Administration Quarterly ,  36 , 585–607.

Deluca, C., Bolden, B., & Chan, J. (2017). Systemic professional learning through collaborative inquiry: Examining teachers’ perspectives.  Teaching and Teacher Education ,  67 , 67–78. https://doi.org/10.1016/j.tate.2017.05.014

Dimmock, C. (1996). Dilemmas for school leaders and administrators in restructuring. In K. Leithwood, J. Chapman, D. Corson, P. Hallinger, & A. Hart (Eds.),  International Handbook of Educational Leadership and Administration  (pp. 135–170). Norwell, MA: Kluwer Academic.

Duke, D. L. (2014). A bold approach to developing leaders for low-performing schools. Management in Education, 28 (3), 80–85. https://doi.org/10.1177/0892020614537665

Gay, L. R., & Airasian, P. (2003).  Educational research: Competancies for analysis and applications  (7th ed.). Upper Saddle River, NJ: Pearson Education.

Glaser, B. G. (1965). The constant comparative method of qualitative analysis.  Social Problems, 12 (4), 436-445.

Hallinger, P., & Bridges, E. (1993). Problem-based learning in medical and managerial education. In P. Hallinger, K. Leithwood, & J. Murphy (Eds.),  Cognitive perspectives on educational leadership  (pp. 253–267). New York: Teachers’ College Press.

Hallinger, P., & Bridges, E. M. (2017). A systematic review of research on the use of problem- based learning in the preparation and development of school leaders.  Educational Administration Quarterly . 53(2), 255-288. https://doi.org/10.1177/0013161X16659347

Hallinger, P., & McCary, C. E. (1990). Developing the strategic thinking of instructional leaders. Elementary School Journal ,  91 (2), 89–108.

Howard, B. B., McClannon, T. W., & Wallace, P. R. (2014). Collaboration through role play among graduate students in educational leadership in distance learning.  American Journal of Distance Education ,  28 (1), 51–61. https://doi.org/10.1080/08923647.2014.868665

Johnson, B. L., & Kruse, S. D. (2009).  Decision making for educational leaders: Underexamined dimensions and issues . Albany, NY: State University of New York Press.

Leithwood, K., & Steinbach, R. (1992). Improving the problem-solving expertise of school administrators: Theory and practice.  Education and Urban Society ,  24 (3), 317–345. Retrieved from https://journals.sagepub.com.ccsu.idm.oclc.org/doi/pdf/10.1177/0013124592024003003

Leithwood, K., & Steinbach, R. (1995).  Expert problem solving: Evidence from school and district leaders . Albany, NY: State University of New York Press.

Marzano, R. J., Waters, T., & McNulty, B. A. (2005).  School leadership that works: From research to results . Denver, CO: ASCD.

Myran, S., & Sutherland, I. (2016). Problem posing in leadership education: Using case study to foster more effective problem solving.  Journal of Cases in Educational Leadership ,  19 (4), 57–71. https://doi.org/10.1177/1555458916664763

Rochford, L., & Borchert, P. S. (2011). Assessing higher level learning: Developing rubrics for case analysis.  Journal of Education for Business ,  86 , 258–265. https://doi.org/10.1080/08832323.2010.512319

Salas, E., Wildman, J. L., & Piccolo, R. F. (2009). Using simulation based training to enhance management education.  Academy of Management Learning & Education ,  8 (4), 559–573. https://doi.org/10.5465/AMLE.2009.47785474

Shapira-Lishchinsky, O. (2015). Simulation-based constructivist approach for education leaders. Educational Management Administration & Leadership ,  43 (6), 972–988. https://doi.org/10.1177/1741143214543203

Simon, H. A. (1993). Decision making: Rational, nonrational, and irrational.  Educational Administration Quarterly ,  29 (3), 392–411. https://doi.org/10.1177/0013161X93029003009

Stentoft, D. (2017). From saying to doing interdisciplinary learning: Is problem-based learning the answer?  Active Learning in Higher Education ,  18 (1), 51–61. https://doi.org/10.1177/1469787417693510

Strauss, A., & Corbin, J. (1998).  Basics of qualitative research  (2nd ed.). Thousand Oaks, CA: Sage.

VanLehn, K. (1991). Rule acquisition events in the discovery of problem-solving strategies. Cognitive Science ,  15 (1), 1–47. https://doi.org/10.1016/0364-0213(91)80012-T

Vygotsky, L. S. (1978).  Mind in society . Cambridge, MA: Harvard University Press.

Winter, R. (1982). Dilemma analysis: A contribution to methodology for action research. Cambridge Journal of Education, 12 (3), 166-173.

Author Biography

Dr. Jeremy Visone is an Assistant Professor of Educational Leadership, Policy, & Instructional Technology. Until 2016, he worked as an administrator at both the elementary and secondary levels, most recently at Anna Reynolds Elementary School, a National Blue Ribbon School in 2016. Dr. Visone can be reached at  [email protected] .

• Position paper
• Open access
• Published: 28 November 2019

Physics education research for 21 st century learning

• Lei Bao   ORCID: orcid.org/0000-0003-3348-4198 1 &
• Kathleen Koenig 2  

Disciplinary and Interdisciplinary Science Education Research volume  1 , Article number:  2 ( 2019 ) Cite this article

37k Accesses

71 Citations

2 Altmetric

Metrics details

Education goals have evolved to emphasize student acquisition of the knowledge and attributes necessary to successfully contribute to the workforce and global economy of the twenty-first Century. The new education standards emphasize higher end skills including reasoning, creativity, and open problem solving. Although there is substantial research evidence and consensus around identifying essential twenty-first Century skills, there is a lack of research that focuses on how the related subskills interact and develop over time. This paper provides a brief review of physics education research as a means for providing a context towards future work in promoting deep learning and fostering abilities in high-end reasoning. Through a synthesis of the literature around twenty-first Century skills and physics education, a set of concretely defined education and research goals are suggested for future research, along with how these may impact the next generation physics courses and how physics should be taught in the future.

Introduction

Education is the primary service offered by society to prepare its future generation workforce. The goals of education should therefore meet the demands of the changing world. The concept of learner-centered, active learning has broad, growing support in the research literature as an empirically validated teaching practice that best promotes learning for modern day students (Freeman et al., 2014 ). It stems out of the constructivist view of learning, which emphasizes that it is the learner who needs to actively construct knowledge and the teacher should assume the role of a facilitator rather than the source of knowledge. As implied by the constructivist view, learner-centered education usually emphasizes active-engagement and inquiry style teaching-learning methods, in which the learners can effectively construct their understanding under the guidance of instruction. The learner-centered education also requires educators and researchers to focus their efforts on the learners’ needs, not only to deliver effective teaching-learning approaches, but also to continuously align instructional practices to the education goals of the times. The goals of introductory college courses in science, technology, engineering, and mathematics (STEM) disciplines have constantly evolved from some notion of weed-out courses that emphasize content drilling, to the current constructivist active-engagement type of learning that promotes interest in STEM careers and fosters high-end cognitive abilities.

Following the conceptually defined framework of twenty-first Century teaching and learning, this paper aims to provide contextualized operational definitions of the goals for twenty-first Century learning in physics (and STEM in general) as well as the rationale for the importance of these outcomes for current students. Aligning to the twenty-first Century learning goals, research in physics education is briefly reviewed to provide a context towards future work in promoting deep learning and fostering abilities in high-end reasoning in parallel. Through a synthesis of the literature around twenty-first Century skills and physics education, a set of concretely defined education and research goals are suggested for future research. These goals include: domain-specific research in physics learning; fostering scientific reasoning abilities that are transferable across the STEM disciplines; and dissemination of research-validated curriculum and approaches to teaching and learning. Although this review has a focus on physics education research (PER), it is beneficial to expand the perspective to view physics education in the broader context of STEM learning. Therefore, much of the discussion will blend PER with STEM education as a continuum body of work on teaching and learning.

Education goals for twenty-first century learning

Education goals have evolved to emphasize student acquisition of essential “21 st Century skills”, which define the knowledge and attributes necessary to successfully contribute to the workforce and global economy of the 21st Century (National Research Council, 2011 , 2012a ). In general, these standards seek to transition from emphasizing content-based drilling and memorization towards fostering higher-end skills including reasoning, creativity, and open problem solving (United States Chamber of Commerce, 2017 ). Initiatives on advancing twenty-first Century education focus on skills that converge on three broad clusters: cognitive, interpersonal, and intrapersonal, all of which include a rich set of sub-dimensions.

Within the cognitive domain, multiple competencies have been proposed, including deep learning, non-routine problem solving, systems thinking, critical thinking, computational and information literacy, reasoning and argumentation, and innovation (National Research Council, 2012b ; National Science and Technology Council, 2018 ). Interpersonal skills are those necessary for relating to others, including the ability to work creatively and collaboratively as well as communicate clearly. Intrapersonal skills, on the other hand, reside within the individual and include metacognitive thinking, adaptability, and self-management. These involve the ability to adjust one’s strategy or approach along with the ability to work towards important goals without significant distraction, both essential for sustained success in long-term problem solving and career development.

Although many descriptions exist for what qualifies as twenty-first Century skills, student abilities in scientific reasoning and critical thinking are the most commonly noted and widely studied. They are highly connected with the other cognitive skills of problem solving, decision making, and creative thinking (Bailin, 1996 ; Facione, 1990 ; Fisher, 2001 ; Lipman, 2003 ; Marzano et al., 1988 ), and have been important educational goals since the 1980s (Binkley et al., 2010 ; NCET, 1987 ). As a result, they play a foundational role in defining, assessing, and developing twenty-first Century skills.

The literature for critical thinking is extensive (Bangert-Drowns & Bankert, 1990 ; Facione, 1990 ; Glaser, 1941 ). Various definitions exist with common underlying principles. Broadly defined, critical thinking is the application of the cognitive skills and strategies that aim for and support evidence-based decision making. It is the thinking involved in solving problems, formulating inferences, calculating likelihoods, and making decisions (Halpern, 1999 ). It is the “reasonable reflective thinking focused on deciding what to believe or do” (Ennis, 1993 ). Critical thinking is recognized as a way to understand and evaluate subject matter; producing reliable knowledge and improving thinking itself (Paul, 1990 ; Siegel, 1988 ).

The notion of scientific reasoning is often used to label the set of skills that support critical thinking, problem solving, and creativity in STEM. Broadly defined, scientific reasoning includes the thinking and reasoning skills involved in inquiry, experimentation, evidence evaluation, inference and argument that support the formation and modification of concepts and theories about the natural world; such as the ability to systematically explore a problem, formulate and test hypotheses, manipulate and isolate variables, and observe and evaluate consequences (Bao et al., 2009 ; Zimmerman, 2000 ). Critical thinking and scientific reasoning share many features, where both emphasize evidence-based decision making in multivariable causal conditions. Critical thinking can be promoted through the development of scientific reasoning, which includes student ability to reach a reliable conclusion after identifying a question, formulating hypotheses, gathering relevant data, and logically testing and evaluating the hypothesis. In this way, scientific reasoning can be viewed as a scientific domain instantiation of critical thinking in the context of STEM learning.

In STEM learning, cognitive aspects of the twenty-first Century skills aim to develop reasoning skills, critical thinking skills, and deep understanding, all of which allow students to develop well connected expert-like knowledge structures and engage in meaningful scientific inquiry and problem solving. Within physics education, a core component of STEM education, the learning of conceptual understanding and problem solving remains a current emphasis. However, the fast-changing work environment and technology-driven world require a new set of core knowledge, skills, and habits of mind to solve complex interdisciplinary problems, gather and evaluate evidence, and make sense of information from a variety of sources (Tanenbaum, 2016 ). The education goals in physics are transitioning towards ability fostering as well as extension and integration with other STEM disciplines. Although curriculum that supports these goals is limited, there are a number of attempts, particularly in developing active learning classrooms and inquiry-based laboratory activities, which have demonstrated success. Some of these are described later in this paper as they provide a foundation for future work in physics education.

Interpersonal skills, such as communication and collaboration, are also essential for twenty-first Century problem-solving tasks, which are often open-ended, complex, and team-based. As the world becomes more connected in a multitude of dimensions, tackling significant problems involving complex systems often goes beyond the individual and requires working with others who are increasingly from culturally diverse backgrounds. Due to the rise of communication technologies, being able to articulate thoughts and ideas in a variety of formats and contexts is crucial, as well as the ability to effectively listen or observe to decipher meaning. Interpersonal skills can be promoted by integrating group-learning experiences into the classroom setting, while providing students with the opportunity to engage in open-ended tasks with a team of peer learners who may propose more than one plausible solution. These experiences should be designed such that students must work collaboratively and responsibly in teams to develop creative solutions, which are later disseminated through informative presentations and clearly written scientific reports. Although educational settings in general have moved to providing students with more and more opportunities for collaborative learning, a lack of effective assessments for these important skills has been a limiting factor for producing informative research and widespread implementation. See Liu ( 2010 ) for an overview of measurement instruments reported in the research literature.

Intrapersonal skills are based on the individual and include the ability to manage one’s behavior and emotions to achieve goals. These are especially important for adapting in the fast-evolving collaborative modern work environment and for learning new tasks to solve increasingly challenging interdisciplinary problems, both of which require intellectual openness, work ethic, initiative, and metacognition, to name a few. These skills can be promoted using instruction which, for example, includes metacognitive learning strategies, provides opportunities to make choices and set goals for learning, and explicitly connects to everyday life events. However, like interpersonal skills, the availability of relevant assessments challenges advancement in this area. In this review, the vast amount of studies on interpersonal and intrapersonal skills will not be discussed in order to keep the main focus on the cognitive side of skills and reasoning.

The purpose behind discussing twenty-first Century skills is that this set of skills provides important guidance for establishing essential education goals for modern society and learners. However, although there is substantial research evidence and consensus around identifying necessary twenty-first Century skills, there is a lack of research that focuses on how the related subskills interact and develop over time (Reimers & Chung, 2016 ), with much of the existing research residing in academic literature that is focused on psychology rather than education systems (National Research Council, 2012a ). Therefore, a major and challenging task for discipline-based education researchers and educators is to operationally define discipline-specific goals that align with the twenty-first Century skills for each of the STEM fields. In the following sections, this paper will provide a limited vision of the research endeavors in physics education that can translate the past and current success into sustained impact for twenty-first Century teaching and learning.

Proposed education and research goals

Physics education research (PER) is often considered an early pioneer in discipline-based education research (National Research Council, 2012c ), with well-established, broad, and influential outcomes (e.g., Hake, 1998 ; Hsu, Brewe, Foster, & Harper, 2004 ; McDermott & Redish, 1999 ; Meltzer & Thornton, 2012 ). Through the integration of twenty-first Century skills with the PER literature, a set of broadly defined education and research goals is proposed for future PER work:

Discipline-specific deep learning: Cognitive and education research involving physics learning has established a rich literature on student learning behaviors along with a number of frameworks. Some of the popular frameworks include conceptual understanding and concept change, problem solving, knowledge structure, deep learning, and knowledge integration. Aligned with twenty-first Century skills, future research in physics learning should aim to integrate the multiple areas of existing work, such that they help students develop well integrated knowledge structures in order to achieve deep leaning in physics.

Fostering scientific reasoning for transfer across STEM disciplines: The broad literature in physics learning and scientific reasoning can provide a solid foundation to further develop effective physics education approaches, such as active engagement instruction and inquiry labs, specifically targeting scientific inquiry abilities and reasoning skills. Since scientific reasoning is a more domain-general cognitive ability, success in physics can also more readily inform research and education practices in other STEM fields.

Research, development, assessment, and dissemination of effective education approaches: Developing and maintaining a supportive infrastructure of education research and implementation has always been a challenge, not only in physics but in all STEM areas. The twenty-first Century education requires researchers and instructors across STEM to work together as an extended community in order to construct a sustainable integrated STEM education environment. Through this new infrastructure, effective team-based inquiry learning and meaningful assessment can be delivered to help students develop a comprehensive skills set including deep understanding and scientific reasoning, as well as communication and other non-cognitive abilities.

The suggested research will generate understanding and resources to support education practices that meet the requirements of the Next Generation Science Standards (NGSS), which explicitly emphasize three areas of learning including disciplinary core ideas, crosscutting concepts, and practices (National Research Council, 2012b ). The first goal for promoting deep learning of disciplinary knowledge corresponds well to the NGSS emphasis on disciplinary core ideas, which play a central role in helping students develop well integrated knowledge structures to achieve deep understanding. The second goal on fostering transferable scientific reasoning skills supports the NGSS emphasis on crosscutting concepts and practices. Scientific reasoning skills are crosscutting cognitive abilities that are essential to the development of domain-general concepts and modeling strategies. In addition, the development of scientific reasoning requires inquiry-based learning and practices. Therefore, research on scientific reasoning can produce a valuable knowledge base on education means that are effective for developing crosscutting concepts and promoting meaningful practices in STEM. The third research goal addresses the challenge in the assessment of high-end skills and the dissemination of effective educational approaches, which supports all NGSS initiatives to ensure sustainable development and lasting impact. The following sections will discuss the research literature that provides the foundation for these three research goals and identify the specific challenges that will need to be addressed in future work.

Promoting deep learning in physics education

Physics education for the twenty-first Century aims to foster high-end reasoning skills and promote deep conceptual understanding. However, many traditional education systems place strong emphasis on only problem solving with the expectation that students obtain deep conceptual understanding through repetitive problem-solving practices, which often doesn’t occur (Alonso, 1992 ). This focus on problem solving has been shown to have limitations as a number of studies have revealed disconnections between learning conceptual understanding and problem-solving skills (Chiu, 2001 ; Chiu, Guo, & Treagust, 2007 ; Hoellwarth, Moelter, & Knight, 2005 ; Kim & Pak, 2002 ; Nakhleh, 1993 ; Nakhleh & Mitchell, 1993 ; Nurrenbern & Pickering, 1987 ; Stamovlasis, Tsaparlis, Kamilatos, Papaoikonomou, & Zarotiadou, 2005 ). In fact, drilling in problem solving may actually promote memorization of context-specific solutions with minimal generalization rather than transitioning students from novices to experts.

Towards conceptual understanding and learning, many models and definitions have been established to study and describe student conceptual knowledge states and development. For example, students coming into a physics classroom often hold deeply rooted, stable understandings that differ from expert conceptions. These are commonly referred to as misconceptions or alternative conceptions (Clement, 1982 ; Duit & Treagust, 2003 ; Dykstra Jr, Boyle, & Monarch, 1992 ; Halloun & Hestenes, 1985a , 1985b ). Such students’ conceptions are context dependent and exist as disconnected knowledge fragments, which are strongly situated within specific contexts (Bao & Redish, 2001 , 2006 ; Minstrell, 1992 ).

In modeling students’ knowledge structures, DiSessa’s proposed phenomenological primitives (p-prim) describe a learner’s implicit thinking, cued from specific contexts, as an underpinning cognitive construct for a learner’s expressed conception (DiSessa, 1993 ; Smith III, DiSessa, & Roschelle, 1994 ). Facets, on the other hand, map between the implicit p-prim and concrete statements of beliefs and are developed as discrete and independent units of thought, knowledge, or strategies used by individuals to address specific situations (Minstrell, 1992 ). Ontological categories, defined by Chi, describe student reasoning in the most general sense. Chi believed that these are distinct, stable, and constraining, and that a core reason behind novices’ difficulties in physics is that they think of physics within the category of matter instead of processes (Chi, 1992 ; Chi & Slotta, 1993 ; Chi, Slotta, & De Leeuw, 1994 ; Slotta, Chi, & Joram, 1995 ). More details on conceptual learning and problem solving are well summarized in the literature (Hsu et al., 2004 ; McDermott & Redish, 1999 ), from which a common theme emerges from the models and definitions. That is, learning is context dependent and students with poor conceptual understanding typically have locally connected knowledge structures with isolated conceptual constructs that are unable to establish similarities and contrasts between contexts.

Additionally, this idea of fragmentation is demonstrated through many studies on student problem solving in physics and other fields. It has been shown that a student’s knowledge organization is a key aspect for distinguishing experts from novices (Bagno, Eylon, & Ganiel, 2000 ; Chi, Feltovich, & Glaser, 1981 ; De Jong & Ferguson-Hesler, 1986 ; Eylon & Reif, 1984 ; Ferguson-Hesler & De Jong, 1990 ; Heller & Reif, 1984 ; Larkin, McDermott, Simon, & Simon, 1980 ; Smith, 1992 ; Veldhuis, 1990 ; Wexler, 1982 ). Expert’s knowledge is organized around core principles of physics, which are applied to guide problem solving and develop connections between different domains as well as new, unfamiliar situations (Brown, 1989 ; Perkins & Salomon, 1989 ; Salomon & Perkins, 1989 ). Novices, on the other hand, lack a well-organized knowledge structure and often solve problems by relying on surface features that are directly mapped to certain problem-solving outcomes through memorization (Chi, Bassok, Lewis, Reimann, & Glaser, 1989 ; Hardiman, Dufresne, & Mestre, 1989 ; Schoenfeld & Herrmann, 1982 ).

This lack of organization creates many difficulties in the comprehension of basic concepts and in solving complex problems. This leads to the common complaint that students’ knowledge of physics is reduced to formulas and vague labels of the concepts, which are unable to substantively contribute to meaningful reasoning processes. A novice’s fragmented knowledge structure severely limits the learner’s conceptual understanding. In essence, these students are able to memorize how to approach a problem given specific information but lack the understanding of the underlying concept of the approach, limiting their ability to apply this approach to a novel situation. In order to achieve expert-like understanding, a student’s knowledge structure must integrate all of the fragmented ideas around the core principle to form a coherent and fully connected conceptual framework.

Towards a more general theoretical consideration, students’ alternative conceptions and fragmentation in knowledge structures can be viewed through both the “naïve theory” framework (e.g., Posner, Strike, Hewson, & Gertzog, 1982 ; Vosniadou, Vamvakoussi, & Skopeliti, 2008 ) and the “knowledge in pieces” (DiSessa, 1993 ) perspective. The “naïve theory” framework considers students entering the classroom with stable and coherent ideas (naïve theories) about the natural world that differ from those presented by experts. In the “knowledge in pieces” perspective, student knowledge is constructed in real-time and incorporates context features with the p-prims to form the observed conceptual expressions. Although there exists an ongoing debate between these two views (Kalman & Lattery, 2018 ), it is more productive to focus on their instructional implications for promoting meaningful conceptual change in students’ knowledge structures.

In the process of learning, students may enter the classroom with a range of initial states depending on the population and content. For topics with well-established empirical experiences, students often have developed their own ideas and understanding, while on topics without prior exposure, students may create their initial understanding in real-time based on related prior knowledge and given contextual features (Bao & Redish, 2006 ). These initial states of understanding, regardless of their origin, are usually different from those of experts. Therefore, the main function of teaching and learning is to guide students to modify their initial understanding towards the experts’ views. Although students’ initial understanding may exist as a body of coherent ideas within limited contexts, as students start to change their knowledge structures throughout the learning process, they may evolve into a wide range of transitional states with varying levels of knowledge integration and coherence. The discussion in this brief review on students’ knowledge structures regarding fragmentation and integration are primarily focused on the transitional stages emerged through learning.

The corresponding instructional goal is then to help students more effectively develop an integrated knowledge structure so as to achieve a deep conceptual understanding. From an educator’s perspective, Bloom’s taxonomy of education objectives establishes a hierarchy of six levels of cognitive skills based on their specificity and complexity: Remember (lowest and most specific), Understand, Apply, Analyze, Evaluate, and Create (highest and most general and complex) (Anderson et al., 2001 ; Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956 ). This hierarchy of skills exemplifies the transition of a learner’s cognitive development from a fragmented and contextually situated knowledge structure (novice with low level cognitive skills) to a well-integrated and globally networked expert-like structure (with high level cognitive skills).

As a student’s learning progresses from lower to higher cognitive levels, the student’s knowledge structure becomes more integrated and is easier to transfer across contexts (less context specific). For example, beginning stage students may only be able to memorize and perform limited applications of the features of certain contexts and their conditional variations, with which the students were specifically taught. This leads to the establishment of a locally connected knowledge construct. When a student’s learning progresses from the level of Remember to Understand, the student begins to develop connections among some of the fragmented pieces to form a more fully connected network linking a larger set of contexts, thus advancing into a higher level of understanding. These connections and the ability to transfer between different situations form the basis of deep conceptual understanding. This growth of connections leads to a more complete and integrated cognitive structure, which can be mapped to a higher level on Bloom’s taxonomy. This occurs when students are able to relate a larger number of different contextual and conditional aspects of a concept for analyzing and evaluating to a wider variety of problem situations.

Promoting the growth of connections would appear to aid in student learning. Exactly which teaching methods best facilitate this are dependent on the concepts and skills being learned and should be determined through research. However, it has been well recognized that traditional instruction often fails to help students obtain expert-like conceptual understanding, with many misconceptions still existing after instruction, indicating weak integration within a student’s knowledge structure (McKeachie, 1986 ).

Recognizing the failures of traditional teaching, various research-informed teaching methods have been developed to enhance student conceptual learning along with diagnostic tests, which aim to measure the existence of misconceptions. Most advances in teaching methods focus on the inclusion of inquiry-based interactive-engagement elements in lecture, recitations, and labs. In physics education, these methods were popularized after Hake’s landmark study demonstrated the effectiveness of interactive-engagement over traditional lectures (Hake, 1998 ). Some of these methods include the use of peer instruction (Mazur, 1997 ), personal response systems (e.g., Reay, Bao, Li, Warnakulasooriya, & Baugh, 2005 ), studio-style instruction (Beichner et al., 2007 ), and inquiry-based learning (Etkina & Van Heuvelen, 2001 ; Laws, 2004 ; McDermott, 1996 ; Thornton & Sokoloff, 1998 ). The key approach of these methods aims to improve student learning by carefully targeting deficits in student knowledge and actively encouraging students to explore and discuss. Rather than rote memorization, these approaches help promote generalization and deeper conceptual understanding by building connections between knowledge elements.

Based on the literature, including Bloom’s taxonomy and the new education standards that emphasize twenty-first Century skills, a common focus on teaching and learning can be identified. This focus emphasizes helping students develop connections among fragmented segments of their knowledge pieces and is aligned with the knowledge integration perspective, which focuses on helping students develop and refine their knowledge structure toward a more coherently organized and extensively connected network of ideas (Lee, Liu, & Linn, 2011 ; Linn, 2005 ; Nordine, Krajcik, & Fortus, 2011 ; Shen, Liu, & Chang, 2017 ). For meaningful learning to occur, new concepts must be integrated into a learner’s existing knowledge structure by linking the new knowledge to already understood concepts.

Forming an integrated knowledge structure is therefore essential to achieving deep learning, not only in physics but also in all STEM fields. However, defining what connections must occur at different stages of learning, as well as understanding the instructional methods necessary for effectively developing such connections within each STEM disciplinary context, are necessary for current and future research. Together these will provide the much needed foundational knowledge base to guide the development of the next generation of curriculum and classroom environment designed around twenty-first Century learning.

Developing scientific reasoning with inquiry labs

Scientific reasoning is part of the widely emphasized cognitive strand of twenty-first Century skills. Through development of scientific reasoning skills, students’ critical thinking, open-ended problem-solving abilities, and decision-making skills can be improved. In this way, targeting scientific reasoning as a curricular objective is aligned with the goals emphasized in twenty-first Century education. Also, there is a growing body of research on the importance of student development of scientific reasoning, which have been found to positively correlate with course achievement (Cavallo, Rozman, Blickenstaff, & Walker, 2003 ; Johnson & Lawson, 1998 ), improvement on concept tests (Coletta & Phillips, 2005 ; She & Liao, 2010 ), engagement in higher levels of problem solving (Cracolice, Deming, & Ehlert, 2008 ; Fabby & Koenig, 2013 ); and success on transfer (Ates & Cataloglu, 2007 ; Jensen & Lawson, 2011 ).

Unfortunately, research has shown that college students are lacking in scientific reasoning. Lawson ( 1992 ) found that ~ 50% of intro biology students are not capable of applying scientific reasoning in learning, including the ability to develop hypotheses, control variables, and design experiments; all necessary for meaningful scientific inquiry. Research has also found that traditional courses do not significantly develop these abilities, with pre-to-post-test gains of 1%–2%, while inquiry-based courses have gains around 7% (Koenig, Schen, & Bao, 2012 ; Koenig, Schen, Edwards, & Bao, 2012 ). Others found that undergraduates have difficulty developing evidence-based decisions and differentiating between and linking evidence with claims (Kuhn, 1992 ; Shaw, 1996 ; Zeineddin & Abd-El-Khalick, 2010 ). A large scale international study suggested that learning of physics content knowledge with traditional teaching practices does not improve students’ scientific reasoning skills (Bao et al., 2009 ).

Aligned to twenty-first Century learning, it is important to implement curriculum that is specifically designed for developing scientific reasoning abilities within current education settings. Although traditional lectures may continue for decades due to infrastructure constraints, a unique opportunity can be found in the lab curriculum, which may be more readily transformed to include hands-on minds-on group learning activities that are ideal for developing students’ abilities in scientific inquiry and reasoning.

For well over a century, the laboratory has held a distinctive role in student learning (Meltzer & Otero, 2015 ). However, many existing labs, which haven’t changed much since the late 1980s, have received criticism for their outdated cookbook style that lacks effectiveness in developing high-end skills. In addition, labs have been primarily used as a means for verifying the physical principles presented in lecture, and unfortunately, Hofstein and Lunetta ( 1982 ) found in an early review of the literature that research was unable to demonstrate the impact of the lab on student content learning.

About this same time, a shift towards a constructivist view of learning gained popularity and influenced lab curriculum development towards engaging students in the process of constructing knowledge through science inquiry. Curricula, such as Physics by Inquiry (McDermott, 1996 ), Real-Time Physics (Sokoloff, Thornton, & Laws, 2011 ), and Workshop Physics (Laws, 2004 ), were developed with a primary focus on engaging students in cognitive conflict to address misconceptions. Although these approaches have been shown to be highly successful in improving deep learning of physics concepts (McDermott & Redish, 1999 ), the emphasis on conceptual learning does not sufficiently impact the domain general scientific reasoning skills necessitated in the goals of twenty-first Century learning.

Reform in science education, both in terms of targeted content and skills, along with the emergence of knowledge regarding human cognition and learning (Bransford, Brown, & Cocking, 2000 ), have generated renewed interest in the potential of inquiry-based lab settings for skill development. In these types of hands-on minds-on learning, students apply the methods and procedures of science inquiry to investigate phenomena and construct scientific claims, solve problems, and communicate outcomes, which holds promise for developing both conceptual understanding and scientific reasoning skills in parallel (Trowbridge, Bybee, & Powell, 2000 ). In addition, the availability of technology to enhance inquiry-based learning has seen exponential growth, along with the emergence of more appropriate research methodologies to support research on student learning.

Although inquiry-based labs hold promise for developing students’ high-end reasoning, analytic, and scientific inquiry abilities, these educational endeavors have not become widespread, with many existing physics laboratory courses still viewed merely as a place to illustrate the physical principles from the lecture course (Meltzer & Otero, 2015 ). Developing scientific ideas from practical experiences, however, is a complex process. Students need sufficient time and opportunity for interaction and reflection on complex, investigative tasks. Blended learning, which merges lecture and lab (such as studio style courses), addresses this issue to some extent, but has experienced limited adoption, likely due to the demanding infrastructure resources, including dedicated technology-intensive classroom space, equipment and maintenance costs, and fully committed trained staff.

Therefore, there is an immediate need to transform the existing standalone lab courses, within the constraints of the existing education infrastructure, into more inquiry-based designs, with one of its primary goals dedicated to developing scientific reasoning skills. These labs should center on constructing knowledge, along with hands-on minds-on practical skills and scientific reasoning, to support modeling a problem, designing and implementing experiments, analyzing and interpreting data, drawing and evaluating conclusions, and effective communication. In particular, training on scientific reasoning needs to be explicitly addressed in the lab curriculum, which should contain components specifically targeting a set of operationally-defined scientific reasoning skills, such as ability to control variables or engage in multivariate causal reasoning. Although effective inquiry may also implicitly develop some aspects of scientific reasoning skills, such development is far less efficient and varies with context when the primary focus is on conceptual learning.

Several recent efforts to enhance the standalone lab course have shown promise in supporting education goals that better align with twenty-first Century learning. For example, the Investigative Science Learning Environment (ISLE) labs involve a series of tasks designed to help students develop the “habits of mind” of scientists and engineers (Etkina et al., 2006 ). The curriculum targets reasoning as well as the lab learning outcomes published by the American Association of Physics Teachers (Kozminski et al., 2014 ). Operationally, ISLE methods focus on scaffolding students’ developing conceptual understanding using inquiry learning without a heavy emphasis on cognitive conflict, making it more appropriate and effective for entry level students and K-12 teachers.

Likewise, Koenig, Wood, Bortner, and Bao ( 2019 ) have developed a lab curriculum that is intentionally designed around the twenty-first Century learning goals for developing cognitive, interpersonal, and intrapersonal abilities. In terms of the cognitive domain, the lab learning outcomes center on critical thinking and scientific reasoning but do so through operationally defined sub-skills, all of which are transferrable across STEM. These selected sub-skills are found in the research literature, and include the ability to control variables and engage in data analytics and causal reasoning. For each targeted sub-skill, a series of pre-lab and in-class activities provide students with repeated, deliberate practice within multiple hypothetical science-based scenarios followed by real inquiry-based lab contexts. This explicit instructional strategy has been shown to be essential for the development of scientific reasoning (Chen & Klahr, 1999 ). In addition, the Karplus Learning Cycle (Karplus, 1964 ) provides the foundation for the structure of the lab activities and involves cycles of exploration, concept introduction, and concept application. The curricular framework is such that as the course progresses, the students engage in increasingly complex tasks, which allow students the opportunity to learn gradually through a progression from simple to complex skills.

As part of this same curriculum, students’ interpersonal skills are developed, in part, through teamwork, as students work in groups of 3 or 4 to address open-ended research questions, such as, What impacts the period of a pendulum? In addition, due to time constraints, students learn early on about the importance of working together in an efficient manor towards a common goal, with one set of written lab records per team submitted after each lab. Checkpoints built into all in-class activities involve Socratic dialogue between the instructor and students and promote oral communication. This use of directed questioning guides students in articulating their reasoning behind decisions and claims made, while supporting the development of scientific reasoning and conceptual understanding in parallel (Hake, 1992 ). Students’ intrapersonal skills, as well as communication skills, are promoted through the submission of individual lab reports. These reports require students to reflect upon their learning over each of four multi-week experiments and synthesize their ideas into evidence-based arguments, which support a claim. Due to the length of several weeks over which students collect data for each of these reports, the ability to organize the data and manage their time becomes essential.

Despite the growing emphasis on research and development of curriculum that targets twenty-first Century learning, converting a traditionally taught lab course into a meaningful inquiry-based learning environment is challenging in current reform efforts. Typically, the biggest challenge is a lack of resources; including faculty time to create or adapt inquiry-based materials for the local setting, training faculty and graduate student instructors who are likely unfamiliar with this approach, and the potential cost of new equipment. Koenig et al. ( 2019 ) addressed these potential implementation barriers by designing curriculum with these challenges in mind. That is, the curriculum was designed as a flexible set of modules that target specific sub-skills, with each module consisting of pre-lab (hypothetical) and in-lab (real) activities. Each module was designed around a curricular framework such that an adopting institution can use the materials as written, or can incorporate their existing equipment and experiments into the framework with minimal effort. Other non-traditional approaches have also been experimented with, such as the work by Sobhanzadeh, Kalman, and Thompson ( 2017 ), which targets typical misconceptions by using conceptual questions to engage students in making a prediction, designing and conducting a related experiment, and determining whether or not the results support the hypothesis.

Another challenge for inquiry labs is the assessment of skills-based learning outcomes. For assessment of scientific reasoning, a new instrument on inquiry in scientific thinking analytics and reasoning (iSTAR) has been developed, which can be easily implemented across large numbers of students as both a pre- and post-test to assess gains. iSTAR assesses reasoning skills necessary in the systematical conduct of scientific inquiry, which includes the ability to explore a problem, formulate and test hypotheses, manipulate and isolate variables, and observe and evaluate the consequences (see www.istarassessment.org ). The new instrument expands upon the commonly used classroom test of scientific reasoning (Lawson, 1978 , 2000 ), which has been identified with a number of validity weaknesses and a ceiling effect for college students (Bao, Xiao, Koenig, & Han, 2018 ).

Many education innovations need supporting infrastructures that can ensure adoption and lasting impact. However, making large-scale changes to current education settings can be risky, if not impossible. New education approaches, therefore, need to be designed to adapt to current environmental constraints. Since higher-end skills are a primary focus of twenty-first Century learning, which are most effectively developed in inquiry-based group settings, transforming current lecture and lab courses into this new format is critical. Although this transformation presents great challenges, promising solutions have already emerged from various research efforts. Perhaps the biggest challenge is for STEM educators and researchers to form an alliance to work together to re-engineer many details of the current education infrastructure in order to overcome the multitude of implementation obstacles.

This paper attempts to identify a few central ideas to provide a broad picture for future research and development in physics education, or STEM education in general, to promote twenty-first Century learning. Through a synthesis of the existing literature within the authors’ limited scope, a number of views surface.

Education is a service to prepare (not to select) the future workforce and should be designed as learner-centered, with the education goals and teaching-learning methods tailored to the needs and characteristics of the learners themselves. Given space constraints, the reader is referred to the meta-analysis conducted by Freeman et al. ( 2014 ), which provides strong support for learner-centered instruction. The changing world of the twenty-first Century informs the establishment of new education goals, which should be used to guide research and development of teaching and learning for present day students. Aligned to twenty-first Century learning, the new science standards have set the goals for STEM education to transition towards promoting deep learning of disciplinary knowledge, thereby building upon decades of research in PER, while fostering a wide range of general high-end cognitive and non-cognitive abilities that are transferable across all disciplines.

Following these education goals, more research is needed to operationally define and assess the desired high-end reasoning abilities. Building on a clear definition with effective assessments, a large number of empirical studies are needed to investigate how high-end abilities can be developed in parallel with deep learning of concepts, such that what is learned can be generalized to impact the development of curriculum and teaching methods which promote skills-based learning across all STEM fields. Specifically for PER, future research should emphasize knowledge integration to promote deep conceptual understanding in physics along with inquiry learning to foster scientific reasoning. Integration of physics learning in contexts that connect to other STEM disciplines is also an area for more research. Cross-cutting, interdisciplinary connections are becoming important features of the future generation physics curriculum and defines how physics should be taught collaboratively with other STEM courses.

This paper proposed meaningful areas for future research that are aligned with clearly defined education goals for twenty-first Century learning. Based on the existing literature, a number of challenges are noted for future directions of research, including the need for:

clear and operational definitions of goals to guide research and practice

concrete operational definitions of high-end abilities for which students are expected to develop

effective assessment methods and instruments to measure high-end abilities and other components of twenty-first Century learning

a knowledge base of the curriculum and teaching and learning environments that effectively support the development of advanced skills

integration of knowledge and ability development regarding within-discipline and cross-discipline learning in STEM

effective means to disseminate successful education practices

The list is by no means exhaustive, but these themes emerge above others. In addition, the high-end abilities discussed in this paper focus primarily on scientific reasoning, which is highly connected to other skills, such as critical thinking, systems thinking, multivariable modeling, computational thinking, design thinking, etc. These abilities are expected to develop in STEM learning, although some may be emphasized more within certain disciplines than others. Due to the limited scope of this paper, not all of these abilities were discussed in detail but should be considered an integral part of STEM learning.

Finally, a metacognitive position on education research is worth reflection. One important understanding is that the fundamental learning mechanism hasn’t changed, although the context in which learning occurs has evolved rapidly as a manifestation of the fast-forwarding technology world. Since learning is a process at the interface between a learner’s mind and the environment, the main focus of educators should always be on the learner’s interaction with the environment, not just the environment. In recent education developments, many new learning platforms have emerged at an exponential rate, such as the massive open online courses (MOOCs), STEM creative labs, and other online learning resources, to name a few. As attractive as these may be, it is risky to indiscriminately follow trends in education technology and commercially-incentivized initiatives before such interventions are shown to be effective by research. Trends come and go but educators foster students who have only a limited time to experience education. Therefore, delivering effective education is a high-stakes task and needs to be carefully and ethically planned and implemented. When game-changing opportunities emerge, one needs to not only consider the winners (and what they can win), but also the impact on all that is involved.

Based on a century of education research, consensus has settled on a fundamental mechanism of teaching and learning, which suggests that knowledge is developed within a learner through constructive processes and that team-based guided scientific inquiry is an effective method for promoting deep learning of content knowledge as well as developing high-end cognitive abilities, such as scientific reasoning. Emerging technology and methods should serve to facilitate (not to replace) such learning by providing more effective education settings and conveniently accessible resources. This is an important relationship that should survive many generations of technological and societal changes in the future to come. From a physicist’s point of view, a fundamental relation like this can be considered the “mechanics” of teaching and learning. Therefore, educators and researchers should hold on to these few fundamental principles without being distracted by the surfacing ripples of the world’s motion forward.

Availability of data and materials

Not applicable.

Abbreviations

American Association of Physics Teachers

Investigative Science Learning Environment

Inquiry in Scientific Thinking Analytics and Reasoning

Massive open online course

New Generation Science Standards

• Physics education research

Science Technology Engineering and Math

Alonso, M. (1992). Problem solving vs. conceptual understanding. American Journal of Physics , 60 (9), 777–778. https://doi.org/10.1119/1.17056 .

Article   Google Scholar  

Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., … Wittrock, M. C. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives, abridged edition . White Plains: Longman.

Ates, S., & Cataloglu, E. (2007). The effects of students’ reasoning abilities on conceptual understandings and problem-solving abilities in introductory mechanics. European Journal of Physics , 28 , 1161–1171.

Bagno, E., Eylon, B.-S., & Ganiel, U. (2000). From fragmented knowledge to a knowledge structure: Linking the domains of mechanics and electromagnetism. American Journal of Physics , 68 (S1), S16–S26.

Bailin, S. (1996). Critical thinking. In J. J. Chambliss (Ed.), Philosophy of education: An encyclopedia , (vol. 1671, pp. 119–123). Routledge.

Bangert-Drowns, R. L., & Bankert, E. (1990). Meta-analysis of effects of explicit instruction for critical thinking. Research report. ERIC Number: ED328614.

Google Scholar  

Bao, L., Cai, T., Koenig, K., Fang, K., Han, J., Wang, J., … Wu, N. (2009). Learning and scientific reasoning. Science , 323 , 586–587. https://doi.org/10.1126/science.1167740 .

Bao, L., & Redish, E. F. (2001). Concentration analysis: A quantitative assessment of student states. American Journal of Physics , 69 (S1), S45–S53.

Bao, L., & Redish, E. F. (2006). Model analysis: Representing and assessing the dynamics of student learning. Physical Review Special Topics-Physics Education Research , 2 (1), 010103.

Bao, L., Xiao, Y., Koenig, K., & Han, J. (2018). Validity evaluation of the Lawson classroom test of scientific reasoning. Physical Review Physics Education Research , 14 (2), 020106.

Beichner, R. J., Saul, J. M., Abbott, D. S., Morse, J. J., Deardorff, D., Allain, R. J., … Risley, J. S. (2007). The student-centered activities for large enrollment undergraduate programs (SCALE-UP) project. Research-Based Reform of University Physics , 1 (1), 2–39.

Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., & Rumble, M. (2010). Draft White paper defining 21st century skills . Melbourne: ACTS.

Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: Handbook 1: Cognitive domain . New York: Longman.

Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn , (vol. 11). Washington, DC: National Academy Press.

Brown, A. (1989). Analogical learning and transfer: What develops? In S. Vosniadu, & A. Ortony (Eds.), Similarity and analogical reasoning , (pp. 369–412). New York: Cambridge U.P.

Chapter   Google Scholar  

Cavallo, A. M. L., Rozman, M., Blickenstaff, J., & Walker, N. (2003). Learning, reasoning, motivation, and epistemological beliefs: Differing approaches in college science courses. Journal of College Science Teaching , 33 (3), 18–22.

Chen, Z., & Klahr, D. (1999). All other things being equal: Acquisition and transfer of the control of variables strategy. Child Development , 70 , 1098–1120.

Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science , 13 (2), 145–182.

Chi, M. T., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science , 5 (2), 121–152.

Chi, M. T., & Slotta, J. D. (1993). The ontological coherence of intuitive physics. Cognition and Instruction , 10 (2–3), 249–260.

Chi, M. T., Slotta, J. D., & De Leeuw, N. (1994). From things to processes: A theory of conceptual change for learning science concepts. Learning and Instruction , 4 (1), 27–43.

Chi, M. T. H. (1992). Conceptual change within and across ontological categories: Examples from learning and discovery in science. In R. N. Giere (Ed.), Cognitive models of science . Minneapolis: University of Minnesota Press.

Chiu, M. H. (2001). Algorithmic problem solving and conceptual understanding of chemistry by students at a local high school in Taiwan. Proceedings-National Science Council Republic of China Part D Mathematics Science and Technology Education , 11 (1), 20–38.

Chiu, M.-H., Guo, C. J., & Treagust, D. F. (2007). Assessing students’ conceptual understanding in science: An introduction about a national project in Taiwan. International Journal of Science Education , 29 (4), 379–390.

Clement, J. (1982). Students’ preconceptions in introductory mechanics. American Journal of Physics , 50 (1), 66–71.

Coletta, V. P., & Phillips, J. A. (2005). Interpreting FCI scores: Normalized gain, preinstruction scores, and scientific reasoning ability. American Journal of Physics , 73 (12), 1172–1182.

Cracolice, M. S., Deming, J. C., & Ehlert, B. (2008). Concept learning versus problem solving: A cognitive difference. Journal of Chemical Education , 85 (6), 873.

De Jong, T., & Ferguson-Hesler, M. G. M. (1986). Cognitive structure of good and poor problem solvers in physics. Journal of Educational Psychology , 78 , 279–288.

DiSessa, A. A. (1993). Toward an epistemology of physics. Cognition and Instruction , 10 (2–3), 105–225.

Duit, R., & Treagust, D. F. (2003). Conceptual change: A powerful framework for improving science teaching and learning. International Journal of Science Education , 25 (6), 671–688.

Dykstra Jr., D. I., Boyle, C. F., & Monarch, I. A. (1992). Studying conceptual change in learning physics. Science Education , 76 (6), 615–652.

Ennis, R. (1993). Critical thinking assessment. Theory Into Practice , 32 (3), 179–186.

Etkina, E., & Van Heuvelen, A. (2001). Investigative science learning environment: Using the processes of science and cognitive strategies to learn physics. In Proceedings of the 2001 physics education research conference , (pp. 17–21). Rochester.

Etkina, E., Van Heuvelen, A., White-Brahmia, S., Brookes, D. T., Gentile, M., Murthy, S., … Warren, A. (2006). Scientific abilities and their assessment. Physical Review Special Topics-Physics Education Research , 2 (2), 020103.

Eylon, B.-S., & Reif, F. (1984). Effects of knowledge organization on task performance. Cognition and Instruction , 1 (1), 5–44.

Fabby, C., & Koenig, K. (2013). Relationship of scientific reasoning to solving different physics problem types. In Proceedings of the 2013 Physics Education Research Conference, Portland, OR .

Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction – The Delphi report . Millbrae: California Academic Press.

Ferguson-Hesler, M. G. M., & De Jong, T. (1990). Studying physics texts: Differences in study processes between good and poor solvers. Cognition and Instruction , 7 (1), 41–54.

Fisher, A. (2001). Critical thinking: An introduction . Cambridge: Cambridge University Press.

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences , 111 (23), 8410–8415.

Glaser, E. M. (1941). An experiment in the development of critical thinking . New York: Teachers College, Columbia University.

Hake, R. R. (1992). Socratic pedagogy in the introductory physics laboratory. The Physics Teacher , 30 , 546.

Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics , 66 (1), 64–74.

Halloun, I. A., & Hestenes, D. (1985a). The initial knowledge state of college physics students. American Journal of Physics , 53 (11), 1043–1055.

Halloun, I. A., & Hestenes, D. (1985b). Common sense concepts about motion. American Journal of Physics , 53 (11), 1056–1065.

Halpern, D. F. (1999). Teaching for critical thinking: Helping college students develop the skills and dispositions of a critical thinker. New Directions for Teaching and Learning , 80 , 69–74. https://doi.org/10.1002/tl.8005 .

Hardiman, P. T., Dufresne, R., & Mestre, J. P. (1989). The relation between problem categorization and problem solving among experts and novices. Memory & Cognition , 17 (5), 627–638.

Heller, J. I., & Reif, F. (1984). Prescribing effective human problem-solving processes: Problem description in physics. Cognition and Instruction , 1 (2), 177–216.

Hoellwarth, C., Moelter, M. J., & Knight, R. D. (2005). A direct comparison of conceptual learning and problem solving ability in traditional and studio style classrooms. American Journal of Physics , 73 (5), 459–462.

Hofstein, A., & Lunetta, V. N. (1982). The role of the laboratory in science teaching: Neglected aspects of research. Review of Educational Research , 52 (2), 201–217.

Hsu, L., Brewe, E., Foster, T. M., & Harper, K. A. (2004). Resource letter RPS-1: Research in problem solving. American Journal of Physics , 72 (9), 1147–1156.

Jensen, J. L., & Lawson, A. (2011). Effects of collaborative group composition and inquiry instruction on reasoning gains and achievement in undergraduate biology. CBE - Life Sciences Education , 10 , 64–73.

Johnson, M. A., & Lawson, A. E. (1998). What are the relative effects of reasoning ability and prior knowledge on biology achievement in expository and inquiry classes? Journal of Research in Science Teaching , 35 (1), 89–103.

Kalman, C., & Lattery, M. (2018). Three active learning strategies to address mixed student epistemologies and promote conceptual change. Frontiers in ICT , 5 (19), 1–9.

Karplus, R. (1964). The science curriculum improvement study. Journal of College Science Teaching , 2 (4), 293–303.

Kim, E., & Pak, S.-J. (2002). Students do not overcome conceptual difficulties after solving 1000 traditional problems. American Journal of Physics , 70 (7), 759–765.

Koenig, K., Schen, M., & Bao, L. (2012). Explicitly targeting pre-service teacher scientific reasoning abilities and understanding of nature of science through an introductory science course. Science Educator , 21 (2), 1–9.

Koenig, K., Schen, M., Edwards, M., & Bao, L. (2012). Addressing STEM retention through a scientific thought and methods course. Journal of College Science Teaching , 41 , 23–29.

Koenig, K., Wood, K., Bortner, L., & Bao, L. (2019). Modifying traditional labs to target scientific reasoning. Journal of College Science Teaching , 48 (5), 28-35.

Kozminski, J., Beverly, N., Deardorff, D., Dietz, R., Eblen-Zayas, M., Hobbs, R., … Zwickl, B. (2014). AAPT recommendations for the undergraduate physics laboratory curriculum , (pp. 1–29). American Association of Physics Teachers Retrieved from https://www.aapt.org/Resources/upload/LabGuidlinesDocument_EBendorsed_nov10.pdf .

Kuhn, D. (1992). Thinking as argument. Harvard Educational Review , 62 (2), 155–178.

Larkin, J., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Expert and novice performance in solving physics problems. Science , 208 (4450), 1335–1342.

Laws, P. W. (2004). Workshop physics activity guide, module 4: Electricity and magnetism. In Workshop physics activity guide . Wiley-VCH.

Lawson, A. E. (1978), The development and validation of a classroom test of formal reasoning, Journal of Research in Science Teaching , 15 (1), 11–24.

Lawson, A. E. (1992). The development of reasoning among college biology students - a review of research. Journal of College Science Teaching , 21 , 338–344.

Lawson, A. E. (2000). Classroom test of scientific reasoning: Multiple choice version, based on Lawson, A. E. 1978. Development and validation of the classroom test of formal reasoning. Journal of Research in Science Teaching , 15 (1), 11–24.

Lee, H. S., Liu, O. L., & Linn, M. C. (2011). Validating measurement of knowledge integration in science using multiple-choice and explanation items. Applied Measurement in Education , 24 (2), 115–136.

Linn, M. C. (2005). The knowledge integration perspective on learning and instruction. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences , (pp. 243–264). Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511816833.016 .

Lipman, M. (2003). Thinking in education , (2nd ed., ). Cambridge: Cambridge University Press.

Liu, X. (2010). Science and engineering education sources. Using and developing measurement instruments in science education: A Rasch modeling approach . Charlotte: IAP Information Age Publishing.

Marzano, R. J., Brandt, R. S., Hughes, C. S., Jones, B. F., Presseisen, B. Z., Rankin, S. C., et al. (1988). Dimensions of thinking, a framework for curriculum and instruction . Alexandria: Association for Supervision and Curriculum Development.

Mazur, E. (1997). Peer instruction: A user’s manual . Upper Saddle River: Prentice Hall.

McDermott, L. C. (1996). Physics by Inquiry: An Introduction to the Physical Sciences . John Wiley & Sons, New York, NY.

McDermott, L. C., & Redish, E. F. (1999). Resource letter: PER-1: Physics education research. American Journal of Physics , 67 (9), 755–767.

McKeachie, W. J. (1986). Teaching and learning in the college classroom: A review of the research literature . Ann Arbor: National Center for Research to Improve Postsecondary Teaching and Learning.

Meltzer, D. E., & Otero, V. K. (2015). A brief history of physics education in the United States. American Journal of Physics , 83 (5), 447–458.

Meltzer, D. E., & Thornton, R. K. (2012). Resource letter ALIP-1: Active-learning instruction in physics. American Journal of Physics , 80 (6), 478–496.

Minstrell, J. (1992). Facets of students’ knowledge and relevant instruction. In R. Duit, F. Goldberg, & H. Niedderer (Eds.), Proceedings of the international workshop: Research in physics learning- theoretical issues and empirical studies , (pp. 110–128). The Institute for Science Education.

Nakhleh, M. B. (1993). Are our students conceptual thinkers or algorithmic problem solvers? Identifying conceptual students in general chemistry. Journal of Chemical Education , 70 (1), 52. https://doi.org/10.1021/ed070p52 .

Nakhleh, M. B., & Mitchell, R. C. (1993). Concept learning versus problem solving: There is a difference. Journal of Chemical Education , 70 (3), 190. https://doi.org/10.1021/ed070p190 .

National Research Council (2011). Assessing 21st century skills: Summary of a workshop . Washington, DC: The National Academies Press. https://doi.org/10.17226/13215 .

Book   Google Scholar  

National Research Council (2012a). Education for life and work: Developing transferable knowledge and skills in the 21st century . Washington, DC: The National Academies Press.

National Research Council (2012b). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas . Washington, DC: National Academies Press.

National Research Council (2012c). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering . Washington, DC: National Academies Press.

National Science & Technology Council (2018). Charting a course for success: America’s strategy for STEM education . Washington, DC: Office of Science and Technology Policy.

NCET. (1987). Critical thinking as defined by the National Council for excellence in critical thinking, statement by Michael Scriven & Richard Paul, presented at the 8th annual conference on critical thinking and education reform. Retrieved December 4, 2018, from http://www.criticalthinking.org/pages/defining-critical-thinking/766 .

Nordine, J., Krajcik, J., & Fortus, D. (2011). Transforming energy instruction in middle school to support integrated understanding and future learning. Science Education , 95 (4), 670–699.

Nurrenbern, S. C., & Pickering, M. (1987). Concept learning versus problem solving: Is there a difference? Journal of Chemical Education , 64 (6), 508.

Paul, R. (1990). Critical thinking: What every person needs to survive in a rapidly changing world . Rohnert Park: Center for Critical Thinking and Moral Critique.

Perkins, D. N., & Salomon, G. (1989). Are cognitive skills context-bound? Educational Researcher , 18 (1), 16–25.

Posner, G., Strike, K., Hewson, P., & Gertzog, W. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education , 66 (2), 211–227.

Reay, N. W., Bao, L., Li, P., Warnakulasooriya, R., & Baugh, G. (2005). Toward the effective use of voting machines in physics lectures. American Journal of Physics , 73 (6), 554–558.

Reimers, F. M., & Chung, C. K. (Eds.) (2016). Teaching and learning for the twenty-first century: Educational goals, policies and curricula from six nations . Cambridge: Harvard Education Press.

Salomon, G., & Perkins, D. N. (1989). Rocky roads to transfer: Rethinking mechanism of a neglected phenomenon. Educational Psychologist , 24 (2), 113–142.

Schoenfeld, A. H., & Herrmann, D. J. (1982). Problem perception and knowledge structure in expert and novice mathematical problem solvers. Journal of Experimental Psychology: Learning, Memory, and Cognition , 8 (5), 484.

Shaw, V. F. (1996). The cognitive processes in informal reasoning. Thinking and Reasoning , 2 (1), 51–80.

She, H., & Liao, Y. (2010). Bridging scientific reasoning and conceptual change through adaptive web-based learning. Journal of Research in Science Teaching , 47 (1), 91–119.

Shen, J., Liu, O. L., & Chang, H.-Y. (2017). Assessing students’ deep conceptual understanding in physical sciences: An example on sinking and floating. International Journal of Science and Mathematics Education , 15 (1), 57–70. https://doi.org/10.1007/s10763-015-9680-z .

Siegel, H. (1988). Educating reason: Rationality, critical thinking and education , (vol. 1). New York: Routledge.

Slotta, J. D., Chi, M. T., & Joram, E. (1995). Assessing students’ misclassifications of physics concepts: An ontological basis for conceptual change. Cognition and Instruction , 13 (3), 373–400.

Smith III, J. P., DiSessa, A. A., & Roschelle, J. (1994). Misconceptions reconceived: A constructivist analysis of knowledge in transition. The Journal of the Learning Sciences , 3 (2), 115–163.

Smith, M. U. (1992). Expertise and organization of knowledge: Unexpected differences among genetic counselors, faculty members and students on problem categorization tasks. Journal of Research in Science Teaching , 29 (2), 179–205.

Sobhanzadeh, M., Kalman, C. S., & Thompson, R. I. (2017). Labatorials in introductory physics courses. European Journal of Physics , 38 , 1–18.

Sokoloff, D. R., Thornton, R. K., & Laws, P. W. (2011). RealTime physics: Active learning laboratories . New York: Wiley.

Stamovlasis, D., Tsaparlis, G., Kamilatos, C., Papaoikonomou, D., & Zarotiadou, E. (2005). Conceptual understanding versus algorithmic problem solving: Further evidence from a national chemistry examination. Chemistry Education Research and Practice , 6 (2), 104–118.

Tanenbaum, C. (2016). STEM 2026: A vision for innovation in STEM education . Washington, DC: US Department of Education.

Thornton, R. K., & Sokoloff, D. R. (1998). Assessing student learning of Newton’s laws: The force and motion conceptual evaluation and the evaluation of active learning laboratory and lecture curricula. American Journal of Physics , 66 (4), 338–352.

Trowbridge, L. W., Bybee, R. W., & Powell, J. C. (2000). Teaching secondary school science: Strategies for developing scientific literacy . Upper Saddle River: Merrill-Prentice Hall.

United States Chamber of Commerce (2017). Bridging the soft skills gap: How the business and education sectors are partnering to prepare students for the 21 st century workforce . Washington DC: Center for Education and Workforce, U.S. Chamber of Commerce Foundation.

Veldhuis, G. H. (1990). The use of cluster analysis in categorization of physics problems. Science Education , 74 (1), 105–118.

Vosniadou, S., Vamvakoussi, X., & Skopeliti, I. (2008). The framework theory approach to the problem of conceptual change. In S. Vosniadou (Ed.), International handbook of research on conceptual change . New York: Routledge.

Wexler, P. (1982). Structure, text, and subject: A critical sociology of school knowledge. In M. W. Apple (Ed.), Cultural and economic reproduction in education: Essays on class, ideology and the state . London: Routledge & Regan Paul.

Zeineddin, A., & Abd-El-Khalick, F. (2010). Scientific reasoning and epistemological commitments: Coordination of theory and evidence among college science students. Journal of Research in Science Teaching , 47 (9), 1064–1093.

Zimmerman, C. (2000). The development of scientific reasoning skills. Developmental Review , 20 (1), 99–149.

Download references

Acknowledgements

The research is supported in part by NSF Awards DUE-1431908 and DUE-1712238. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

The research is supported in part by NSF Awards DUE-1431908 and DUE-1712238.

Author information

Authors and affiliations.

The Ohio State University, Columbus, OH, 43210, USA

University of Cincinnati, Cincinnati, OH, 45221, USA

Kathleen Koenig

You can also search for this author in PubMed   Google Scholar

Contributions

LB developed the concept, wrote a significant portion of the review and position, and synthesized the paper. KK wrote and edited a significant portion of the paper. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Lei Bao .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Bao, L., Koenig, K. Physics education research for 21 st century learning. Discip Interdscip Sci Educ Res 1 , 2 (2019). https://doi.org/10.1186/s43031-019-0007-8

Download citation

Received : 17 April 2019

Accepted : 13 June 2019

Published : 28 November 2019

DOI : https://doi.org/10.1186/s43031-019-0007-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Twenty-first century learning
• STEM education
• Scientific reasoning
• Deep learning

How can procedural flowcharts support the development of mathematics problem-solving skills?

• Original Article
• Open access
• Published: 22 February 2024

Cite this article

You have full access to this open access article

• Musarurwa David Chinofunga   ORCID: orcid.org/0000-0002-0262-3039 1 ,
• Philemon Chigeza   ORCID: orcid.org/0000-0001-9964-0988 1 &
• Subhashni Taylor   ORCID: orcid.org/0000-0002-1624-0901 1  

599 Accesses

1 Altmetric

Explore all metrics

Supporting students’ problem-solving skills, solution planning and sequencing of different stages that are involved in successfully developing a meaningful solution to a problem has been a challenge for teachers. This case study was informed by reflective investigation methodology which explored how procedural flowcharts can support student mathematics problem solving in a senior Mathematical Methods subject in Queensland. The paper used thematic analysis to analyse and report on teachers’ perceptions of the utility of procedural flowcharts during problem solving as well as content analysis on how student-developed flowcharts can support their problem-solving skills. Results show that development of procedural flowcharts can support problem solving as it helps with integration of problem-solving stages.

Similar content being viewed by others

Students’ individual schematization pathways - empirical reconstructions for the case of part-of-part determination for fractions.

Matthias Glade & Susanne Prediger

Identifying Metacognitive Behavior in Problem-Posing Processes

Lukas Baumanns & Benjamin Rott

Sequencing of Practice Examples for Mathematical Reasoning: A Case of a Singapore Secondary School Teacher’s Practice

Avoid common mistakes on your manuscript.

Introduction

Problem solving is central to teaching and learning of mathematics (see Cai, 2010 ; Lester, 2013 ; Schoenfeld et al., 2014 ). For decades, research in mathematics problem solving, including special issues from leading mathematics education journals (see, Educational Studies in Mathematics, (Vol. 83, no. 2013); The Mathematics Enthusiast, (Vol. 10, nos. 1–2); ZDM , (Vol. 39, nos. 5–6)), have offered significant insights but struggled to produce well-articulated guidelines for educational practice (English & Gainsburg, 2016 ). This could possibly be the reason why mathematics teachers’ efforts to improve students’ problem-solving skills have not produced the desired results (Anderson, 2014 ; English & Gainsburg, 2016 ). Despite Polya’s ( 1945 ) heuristics being so valuable in problem solving, there appears to be limited success when translated into the classroom environment (English & Gainsburg, 2016 ). English and Gainsburg went further to posit that one of the issues to be addressed is how to support problem-solving competency in students during the process of problem solving. Thus, teachers’ perceptions in this study are a valuable part in evaluating how procedural flowcharts can support problem solving.

The problem-solving process is a dialogue between the prior knowledge the problem solver possesses, the tentative plan of solving the problem and other relevant thoughts and facts (Schoenfeld, 1983 ). However, research is still needed on tools that teachers can use to support students during problem solving (Lester & Cai, 2016 ). Although research in mathematics problem solving has been progressing, it has remained largely theoretical (Lester, 2013 ). Schoenfeld ( 2013 ) suggests that research focus should now advance from the framework for examining problem solving to explore how ideas grow and are presented and shared during the problem-solving process. Recently, Kaitera and Harmoinen ( 2022 ) emphasised the need to support teachers through resources that can help students develop problem solving skills. They went on to posit that resources that can assist students in presenting different approaches to a solution and displaying their understanding are critical to build their problem-solving skills.

The study by Kaitera and Harmoinen ( 2022 ) introduced mathematics students to ‘problem-solving keys’ which are heuristics for problem solving that students are to follow as they engage with tasks. Their conclusion was also noted by Vale and Barbosa ( 2018 ) who observed that a key area that would benefit from further research is the identification of strategies or plan that support students’ ability to construct and present their mathematical knowledge effectively during problem solving, particularly if complex processes such as integration and modification of several procedures are involved. Similarly, students face challenges in connecting or bringing all the ideas together and showing how they relate as they work towards the solution (Reinholz, 2020 ). Problem solving in mathematics is challenging for students (Ahmad et al., 2010 ), and therefore, supporting students’ problem-solving skills needs urgent attention (Schoenfeld, 2016 ). Furthermore, Mason ( 2016 ) posits that the crucial yet not significantly understood issue for adopting a problem-solving approach to teaching is the issue of “when to introduce explanatory tasks, when to intervene and in what way” (p. 263). Therefore, teachers also need resources to support the teaching of problem-solving skills, often because they were not taught these approaches when they were school students (Kaitera & Harmoinen, 2022 ; Sakshaug & Wohlhuter, 2010 ).

Flowcharts have been widely used in problem solving across different fields. In a technology-rich learning environment such as Lego Robotics, creating flowcharts to explain processes was observed to facilitate understanding, thinking, making sense of how procedures relate, investigating and communicating the solution (Norton et al., 2007 ). They are effective in guiding students during problem solving (Gencer, 2023 ), enhancing achievement and improving problem-solving skills in game-based intelligent tutoring (Hooshyar et al., 2016 ). Flowcharts have been identified as an effective problem-solving tool in health administration (McGowan & Boscia, 2016 ). In mathematics education, heuristic trees and flowcharts were observed to supplement each other in influencing students’ problem solving behaviour (Bos & van den Bogaart, 2022 ). Importantly, McGowan and Boscia emphasised that “one of the greatest advantages of a flowchart is its ability to provide for the visualisation of complex processes, aiding in the understanding of the flow of work, identifying nonvalue-adding activities and areas of concern, and leading to improved problem-solving and decision-making” (p. 213). Identifying the most appropriate strategy and making the correct decision at the right stage are keys to problem solving. Teaching students to use visual representations like flowcharts as part of problem solving supports the ability to easily identify new relationships among different procedures and assess the solution being communicated faster as visual representations are more understandable (Vale et al., 2018 ).

The purpose of this case study was to explore, through an in-depth teacher’s interview, and student-developed artefacts, the utility of procedural flowcharts in supporting the development of students’ problem-solving skills in mathematics. The study will focus on problem solving in Mathematical Methods which is one of the calculus-based mathematics subjects at senior school in Queensland. The aim was to investigate if the development of procedural flowcharts supported students in planning, logically connecting and integrating mathematical procedures (knowledge) and to communicate the solution effectively during problem solving. The use of flowcharts in this study was underpinned by the understanding that visual aids that support cognitive processes and interlinking of ideas and procedures influence decision-making, which is vital in problem-based learning (McGowan & Boscia, 2016 ). Moreover, flowcharts are effective tools for communicating the processes that need to be followed in problem solving (Krohn, 1983 ).

Problem-solving learning in mathematics education

The drive to embrace a problem-solving approach to develop and deepen students’ mathematics knowledge has been a priority in mathematics education (Koellner et al., 2011 ; Sztajn et al., 2017 ). In the problem-solving approach, the teacher provides the problem to be investigated by students who then design ways to solve it (Colburn, 2000 ). To engage in problem solving, students are expected to use concepts and procedures that they have learnt (prior knowledge) and apply them in unfamiliar situations (Matty, 2016 ). Teachers are encouraged to promote problem-solving activities as they involve students engaging with a mathematics task where the procedure or method to the solution is not known in advance (National Council of Teachers of Mathematics [NCTM], 2000 ), thus providing opportunities for deep understanding as well as providing students with the opportunity to develop a unique solution (Queensland Curriculum and Assessment Authority [QCAA], 2018 ). Using this approach, students are given a more active role through applying and adapting procedures to solve a non-routine problem and then communicating the method (Karp & Wasserman, 2015 ). The central role problem solving plays in developing students’ mathematical understanding has resulted in the development of different problem-solving models over the years.

The process of problem solving in mathematics requires knowledge to be organised as the solution is developed and then communicated. Polya is among the first to systematise problem solving in mathematics (Voskoglou, 2021 ). Students need to understand the problem, plan the solution, execute the plan and reflect on the solution and process (Polya, 1971 ). Voskoglou’s ( 2021 ) problem-solving model emphasised that the process of modelling involves analysis of problem, mathematisation, solution development, validation and implementation. Similarly, problem solving is guided by four phases: discover, devise, develop and defend (Makar, 2012 ). During problem solving, students engage with an unfamiliar real-world problem, develop plans in response, justify mathematically through representation, then evaluate and communicate the solution (Artigue & Blomhøj, 2013 ). Furthermore, Schoenfeld ( 1980 ) posited that problem solving involves problem analysis, exploration, design, implementation and verification of the solution. When using a problem-solving approach, students can pose questions, develop way(s) to answer problems (which might include drawing diagrams, carrying out calculations, defining relationships and making conclusions), interpreting, evaluating and communicating the solution (Artigue et al., 2020 ; Dorier & Maass, 2020 ). Problem solving involves understanding the problem, devising and executing the plan and evaluating (Nieuwoudt, 2015 ). Likewise, Blum and Leiß ( 2007 ) developed a modelling approach that was informed by these stages, understanding, simplifying, mathematising, working mathematically, interpreting and validating.

Similarly, mathematical modelling involves problem identification from a contextualised real-world problem, linking the solution to mathematics concepts, carrying out mathematic manipulations, justifying and evaluating the solution in relation to the problem and communicating findings (Geiger et al., 2021 ). Likewise, in modelling, Galbraith and Stillman ( 2006 ) suggested that further research is needed in fostering students’ ability to transition effectively from one phase to the next. “Mathematical modelling is a special kind of problem solving which formulates and solves mathematically real-world problems connected to science and everyday life situations” (Voskoglou, 2021 p. 85). As part of problem solving, mathematical modelling requires students to interpret information from a variety of narrative, expository and graphic texts that reflect authentic real-life situations (Doyle, 2005 ).

There are different approaches to problem solving and modelling, but all of them focus on the solving of real-world problems using mathematical procedures and strategies (Hankeln, 2020 ). A literature synthesis is critical where several models exist as it can be used to develop an overarching conceptual model (Snyder, 2019 ). Torraco ( 2005 ) noted that literature synthesis can be used to integrate different models that address the same phenomenon. For example, in this study, it was used to integrate problem solving models cited in the literature. Moreover, the review was necessitated by the need to reconceptualise the problem-solving model by Polya ( 1971 ) to include the understanding that the definition of problem solving has now broadened to include modelling. Torraco went further to suggest that as literature grows, and knowledge expands on a topic which might accommodate new insights, there is a need for literature synthesis with the aim to reflect the changes. Thus, the model in Fig.  1 took into consideration the key stages broadly identified by the researchers and the understanding that modelling is part of problem solving. Problem solving and modelling is generally a linear process that can include loops depending on how the problem identification, mathematisation and implementation effectively address the problem (Blum & Leiß, 2007 ; Polya, 1957 ).

Stages of mathematics problem solving

Figure  1 identifies the main stages that inform mathematics problem solving from the literature.

Problem identification and the design to solve the problem might be revisited if the procedures that were identified and their mathematical justification do not address the problem. Likewise, justification and evaluation after implementation might prompt the problem solver to realise that the problem was incorrectly identified. The loop is identified by the backward arrow, and the main problem-solving stages are identified by the linear arrows. The Australian Curriculum, Assessment and Reporting Authority notes that during problem solving:

Students solve problems when they use mathematics to represent unfamiliar situations, when they design investigations and plan their approaches, when they apply their existing knowledge to seek solutions, and when they verify that their answers are reasonable. Students develop the ability to make choices, interpret, formulate, model and investigate problem situations, and communicate solutions effectively. (Australia Curriculum and Reporting Authority, 2014 , p. 5)

Therefore, during problem solving, students have to plan the solution to the problem and be able to communicate all the key processes involved. However, although problem solving is highly recommended in mathematics education, it presents several challenges for teachers in terms of how they can best support students to connect the processes and mathematics concepts into something coherent that can lead to a meaningful solution (Hacker, 1998 ). Therefore, relevant tools that support problem solving and decision-making can make a difference for both mathematics teachers and students (McGowan & Boscia, 2016 ).

Students can solve problems better if they can think critically (Kules, 2016 ). Problem solving requires their active engagement in analysing, conceptualising, applying concepts, evaluating, comparing, sequencing, synthesising, reasoning, reflecting and communicating, which are skills that are said to promote critical thinking (Kim et al., 2012 ; King, 1995 ; Moon, 2008 ; QCAA, 2018 ). Similarly, the ability to undertake problem solving is supported when students are provided with the opportunity to sequence ideas logically and evaluate the optimal strategy to solve the problem (Parvaneh & Duncan, 2021 ). However, finding tools that can support problem solving has been a focus for researchers for a long time but with very limited breakthroughs (McCormick et al., 2015 ). This study explored how procedural flowcharts as visual representations can support students in organising ideas, execute procedures, justify solutions and communicate their solution.

Importance of visual representations in mathematics problem-solving

Research on how visual representations support mathematics discovery and structural thinking in problem solving has come a long way (see Hadamard, 1945 ; Krutetskii, 1976 ; Polya, 1957 ). Visual representations are classified as graphs, tables, maps, diagrams, networks and icons and are widely used to convey information in a recognisable form that can be easily interpreted without resorting to tedious computations (Lohse et al., 1994 ). Visual representations can be used as a tool to capture mathematics relations and processes (van Garderen et al., 2021 ) and used in many cognitive tasks such as problem solving, reasoning and decision making (Zhang, 1997 ). Indeed, representations can be modes of communicating during concepts exploration and problem solving (Roth & McGinn, 1998 ). Likewise, visual representations can be a powerful way of presenting the solution to a problem, including self-monitoring on how the problem is being solved (Kingsdorf & Krawec, 2014 ; Krawec, 2014 ). Using visualisations created by teachers or students in mathematics can support students’ problem-solving abilities (Csíkos et al., 2012 ).

Visual representations show thoughts in non-linguistic format, which is effective for communication and reflection. “Visual representations serve as tools for thinking about and solving problems. They also help students communicate their thinking to others” (NCTM, 2000 , p. 206). In mathematics, visual representation plays a significant role in showing the cognitive constructs of the solution (Owens & Clements, 1998 ), a view echoed by Arcavi ( 2003 ), who said that visual representations can be appreciated as a central part of reasoning and as a resource to use in problem solving. More importantly, they can be used to represent the logical progression of ideas and reasoning during problem solving (Roam, 2009 ). However, there is need to explore how visual representations can be used to support and illustrate the problem-solving process and to create connections among concepts (Stylianou, 2010 ). Importantly, developing diagrams is often a recommended strategy for solving mathematics problems (Pape & Tchoshanov, 2001 ; Jitendra et al., 2013 ; Zahner & Corter, 2010 ). Therefore, this study will explore the utility of procedural flowcharts as a visual representation and resource in supporting problem analysis, problem understanding, solution development and evaluation, while communicating the whole problem-solving process effectively. It will go further to explore how development of procedural flowcharts can support educational practice in Mathematical Methods subject.

Procedural flowcharts are a visual representation of procedures, corresponding steps and stages of evaluation of a solution to a problem (Chinofunga et al., 2022 ). These authors noted that procedural flowcharts developed by the teacher can guide students during the inquiry process and highlight key procedures and stages for decision-making during the process of problem solving. This is because “a procedural flowchart graphically displays the information–decision–action sequences in the proposed order” (Krohn, 1983 , p. 573). Similarly, Chinofunga and colleagues ( 2022 ) emphasised that procedural flowcharts can be used to visually represent procedural flexibility as more than one procedure can be accommodated, making it easier to compare the effectiveness of different procedures as they are being applied. They further posited that student-developed procedural flowcharts provide students with the opportunity to comprehensively engage with the problem and brainstorm different ways of solving it, thus deepening their mathematics knowledge. Moreover, a procedural flowchart can be a visual presentation of an individual or group solution during problem solving.

Research has identified extended benefits of problem solving in small groups (Laughlin et al., 2006 ). Giving groups an opportunity to present a solution visually can be a quicker way to evaluate a group solution because visual representations can represent large amounts of information (even from different sources) in a simple way (Raiyn, 2016 ). Equally, Vale and colleagues encouraged visual representation of solutions with multisolutions as a tool to teach students problem solving ( 2018 ). Therefore, students can be asked to develop procedural flowcharts individually then come together to synthesise different procedural flowcharts.

Similarly, flowcharts are a visual aid used to represent how procedures interrelate and function together. “They are tools to visually break down complex information into individual building blocks and how the blocks are connected” (Grosskinsky et al., 2019 , p. 24). They outlay steps in a procedure and show how they can be applied, thus helping to visualise the process (Ledin & Machin, 2020 ; Reingewertz, 2013 ). Flowcharts can also be used when a logical and sequenced approach is needed to address a problem (Cantatore & Stevens, 2016 ). Importantly, in schools, Norton and colleagues ( 2007 ) noted that “planning facilitated through the use of flow charts should be actively encouraged and scaffolded so that students can appreciate the potential of flow charts to facilitate problem-solving capabilities” (p. 15). This was because the use of flowcharts in problem solving provided a mental representation of a proposed approach to solve a task (Jonassen, 2012 ). The success of flowcharts in problem solving in different fields can be attributed to their ability to facilitate deep engagement in planning the solution to the problem.

Flowcharts use has distinct advantages that can benefit problem solving. Norton and colleagues ( 2007 ) posited that using a well-planned and well-constructed flowchart in problem solving results in a good-quality solution. Furthermore, flowcharts can also be a two-way communication resource between a teacher and students or among students (Grosskinsky et al., 2019 ). These authors further noted that flowcharts can help in checking students’ progress, tracking their progress and guide them. They can also be used to highlight important procedures that students can follow during the process of problem solving.

Similarly, flowcharts can be used to provide a bigger picture of the solution to a problem (Davidowitz & Rollnick, 2001 ). Flowcharts help students gain an overall and coherent understanding of the procedures involved in solving the problem as they promote conceptual chunking (Norton et al., 2007 ). Importantly, “they may function to amplify the zone of proximal development for students by simplifying tasks in the zone” (Davidowitz & Rollnick, 2001 , p. 22). Use of flowcharts by students reduces the cognitive load which then may help them focus on more complex tasks (Berger, 1998 ; Sweller et al., 2019 ). Indeed, development of problem-solving skills can be supported when teachers introduce learning tools such as flowcharts, because they can help structure how the solution is organised (Santoso & Syarifuddin, 2020 ). Therefore, the use of procedural flowcharts in mathematics problem solving has the potential to transform the process.

The research question in this study was informed by the understanding that limited resources are available to teachers to support students’ problem-solving abilities. In addition, the literature indicates that visual representation such as procedural flowcharts can support students’ potential in problem solving. Therefore, the research described in this study addressed the following research question: What are teachers’ perceptions of how procedural flowcharts can support the development of students’ problem-solving skills in the Mathematical Methods subject?

Methodology

The case study draws from the reflective investigation methodology (Trouche et al., 2018 ,  2020 ). The methodology explores how teaching and learning was supported by facilitating a teacher’s reflection on the unexpected use of a resource, in this case procedural flowcharts. The reflective methodology emphasises a teacher’s active participation through soliciting views on the current practice and recollection on previous work (Trouche et al., 2020 ). Using the methodology, a teacher is asked to reflect on and describe the resource used, the structure (related to the activity), the implementation and the outcomes (Huang et al., 2023 ).

This case study focuses on phases three and four of a broad PhD study that involved four phases. The broad study was informed by constructivism. Firstly, phase one investigated Queensland senior students’ mathematics enrolment in different mathematics curricula options from 2010 to 2020. Secondly, phase two developed and introduced pedagogical resources that could support planning, teaching and learning of calculus-based mathematics with a special focus on functions in mathematical methods. The pedagogical resources included a framework on mathematics content sequencing which was developed through literature synthesis to guide teachers on how to sequence mathematics content during planning. Furthermore, the phase also introduced concept maps as a resource for linking prior knowledge to new knowledge in a constructivist setting. Procedural flowcharts were also introduced to teachers in this phase as a resource to support development of procedural fluency in mathematics. Importantly, a conference workshop organised by the Queensland Association of Mathematics Teachers (Cairns Region) provided an opportunity for teachers to contribute their observations on ways that concept maps and procedural flowcharts can be used to support teaching. Thirdly, phase three was a mixed-method study that focused on evaluating the pedagogical resources that were developed or introduced in phase two with 16 purposively sampled senior mathematics teachers in Queensland who had been given a full school term to use the resources in their practice. Some qualitative data collected through semistructured interviews from phase three were included in the results of the study reported here. During the analysis of the qualitative data, a new theme emerged which pointed to the unexpected use of procedural flowcharts during teaching and learning beyond developing procedural fluency. As a result, the researchers decided to explore how development of procedural flowcharts supported teaching and learning of mathematics as an additional phase. Phase four involved an in-depth interview with Ms. Simon (pseudonym) a teacher who had unexpectedly applied procedural flowcharts in a problem-solving task, which warranted further investigation. Ms. Simon’s use of procedural flowcharts was unexpected as she had used them outside the context and original focus of the broader study. Importantly, in phase four, artefacts created by the teacher and her four students in the problem-solving task were also collected.

Ms. Simon (pseudonym) had explored the use of procedural flowcharts in a problem-solving and modelling task (PSMT) in her year 11 Mathematical Methods class. This included an introduction to procedural flowcharts, followed by setting the students a task whereby they were asked to develop a procedural flowchart as an overview on how they would approach a problem-solving task. The students were expected to first develop the procedural flowcharts independently then to work collaboratively to develop and structure an alternative solution to the same task. The student-developed procedural flowcharts (artefacts) and the in-depth interview with Ms. Simon were included in the analysis. As this was an additional study, an ethics amendment was applied for and granted by the James Cook University Ethics committee, approval Number H8201, as the collection of students’ artefacts was not covered by the main study ethics approval for teachers.

Research context of phase four of the study

In the state of Queensland, senior mathematics students engage with three formal assessments (set by schools but endorsed by QCAA) in year 12 before the end of year external examination. The formal internal assessments consist of two written examinations and a problem-solving and modelling task (PSMT). The PSMT is expected to cover content from Unit 3 (Further Calculus). The summative external examination contributes 50% and the PSMT 20% of the overall final mark, demonstrating that the PSMT carries the highest weight among the three formal internal assessments.

The PSMT is the first assessment in the first term of year 12 and is set to be completed in 4 weeks. Students are given 3 h of class time to work on the task within the 4 weeks and write a report of up to 10 pages or 2000 words. The 4 weeks are divided into four check points, one per week with the fourth being the submission date. On the other three checkpoints, students are expected to email their progress to the teacher. At checkpoint one, the student will formulate a general plan on how to solve the problem which is detailed enough for the teacher to provide meaningful feedback. Checkpoint one is where this study expects teachers to provide students with the opportunity to develop a procedural flowchart of the plan to reach the solution. Importantly at checkpoint one, teachers are interested in understanding which mathematics concepts students will select and apply to try and solve the problem and how the concepts integrate or complement each other to develop a mathematically coherent, valid and appropriate solution. Moreover, teachers are expected to have provided students with opportunities to develop skills in undertaking problem-solving and modelling task before they engage with this formal internal assessment. The QCAA has provided a flowchart to guide teachers and students on how to approach a PSMT ( Appendix 1 )

Participants in phase four of the study

Ms. Simon and a group of four students were the participants in this study. Ms. Simon had studied mathematics as part of her undergraduate education degree, which set her as a highly qualified mathematics teacher. At the time of this study, she was the Head of Science and Mathematics and a senior mathematics teacher at one of the state high schools in Queensland. She had 35 years’ experience in teaching mathematics across Australia in both private and state schools, 15 of which were as a curriculum leader. She was also part of the science, technology, engineering and mathematics (STEM) state-wide professional working group. Since the inception of the external examination in Queensland in 2020, she had been an external examination marker and an assessment endorser for Mathematical Methods with QCAA. The students who were part of this study were aged between 17 and 18 years and were from Ms. Simon’s Mathematical Methods senior class. Two artefacts were from individual students, and the third was a collaborative work from the two students.

Phase four data collection

First, data were collected through an in-depth interview between the researcher and Ms. Simon. The researcher used pre-prepared questions and incidental questions arising from the interview. The questions focused on exploring how she had used procedural flowcharts in a PSMT with her students. The interview also focused on her experiences, observations, opinions, perceptions and results, comparing the new experience with how she had previously engaged her students in such tasks. The interview lasted 40 min, was transcribed and coded so as to provide evidence of the processes involved in the problem solving. Some of the pre-prepared questions were as follows:

What made you consider procedural flowcharts as a resource that can be used in a PSMT?

How have you used procedural flowcharts in PSMT?

How has the use of procedural flowcharts transformed students’ problem-solving skills?

How have you integrated procedural flowcharts to complement the QCAA flowchart on PSMT in mathematics?

What was your experience of using procedural flowcharts in a collaborative setting?

How can procedural flowcharts aid scaffolding of problem-solving tasks?

Second, Ms. Simon shared her formative practice PSMT task (described in detail below), and three of her students’ artefacts. The artefacts that she shared (with the students’ permission) were a critical source of data as they were a demonstration of how procedural flowcharts produced by students can support the development of problem solving and provided an insight into the use of procedural flowcharts in a PSMT.

Problem-solving and assessment task

The formative practice PSMT that Ms. Simon shared is summarised below under the subheadings: Scenario, Task, Checkpoints and Scaffolding.

You are part of a team that is working on opening a new upmarket Coffee Café. Your team has decided to cater for mainly three different types of customers. Those who:

Consume their coffee fast.

Have a fairly good amount of time to finish their coffee.

Want to drink their coffee very slowly as they may be reading a book or chatting.

The team has tasked you to come up with a mode or models that can be used to understand the cooling of coffee in relation to the material the cup is made from and the temperature of the surroundings.

Write a mathematical report of at most 2000 words or up to 10 pages that explains how you developed the cooling model/s and took into consideration the open cup, the material the cup was made from, the cooling time, the initial temperature of the coffee and the temperature of the surroundings.

Design an experiment that investigates the differences in the time of cooling of a liquid in open cups made from different materials. Record your data in a table.

Develop a procedural flowchart that shows the steps that you used to arrive at a solution for the problem.

Justify your procedures and decisions by explaining mathematical reasoning.

Provide a mathematical analysis of formulating and evaluating models using both mathematical manipulation and technology.

Provide a mathematical analysis that involves differentiation (rate of change) and/or anti-differentiation (area under a curve) to satisfy the needs of each category of customers.

Evaluate the reasonableness of solutions.

You must consider Newton’s Law of Cooling which states that the rate of change of the temperature of an object is proportional to the difference between its own temperature and the temperature of its surroundings. For a body that has a higher temperature than its surroundings, Newton’s Law of Cooling can model the rate at which the object is cooling in its surroundings through an exponential equation. This equation can be used to model any object cooling in its surroundings: 

y is the difference between the temperature of the body and its surroundings after t minutes,

A 0 is the difference between the initial temperature of the body and its surroundings,

k is the cooling constant.

Checkpoints

Week 1: Students provide individual data from the experiment and create a procedural flowchart showing the proposed solution to the problem. Teacher provides individual feedback. Week 2: Students provide a consolidated group procedural flowchart. Teacher provides group feedback Week 3: Students email a copy of their individually developed draft report for feedback. Week 4: Students submit individual final response in digital (PDF format) by emailing a copy to their teacher, providing a printed copy to their teacher and saving a copy in their Maths folder.

Additional requirements/instructions

The response must be presented using an appropriate mathematical genre (i.e., a mathematical report).

The approach to problem-solving and mathematical modelling must be used.

All sources must be referenced.

Data analysis

The analysis of data includes some observations and perceptions of mathematics teachers which were collected through surveys and interviews from phase three of the broader PhD study. The survey and interviews data in the broader study including phase four in-depth interview with Ms. Simon were transcribed and coded using thematic analysis (TA). TA is widely used in qualitative research to identify and describe patterns of meaning within data (Braun & Clarke, 2006 ; Ozuem et al., 2022 ). The thematic validity was ensured using theory triangulation. It involves sharing qualitative responses among colleagues at different status positions in the field and then comparing findings and conclusions (Guion et al., 2011 ). The study adopted the inductive approach which produces codes that are solely reflective of the contents of the data (Byrne, 2022 ).

Coding was done with no pre-set codes, and line-by-line coding was used as this was mainly an inductive analysis. The research team comprising of the researcher and two advisors/supervisors met to set the initial coding mechanism and code part of the data for consistency before independent coding of all the data. This is supported by King ( 2004 ) who suggested that when searching for themes, it is best to start with a few codes to help guide analysis. The data covered a wide variety of concepts, so initially the different concepts that grouped the research questions as ‘conceptual themes’ were utilised to organise the data. The research team examined the codes, checking their meaning and relationships between them to determine which ones were underpinned by a central concept. In Excel, codes that shared a core idea from the initial phase that used data from the open-ended responses and interview transcripts were colour coded. After the independent thematic analysis, the filter function in Excel was used to sort the codes using cell colour. Moreover, Excel provided the opportunity to identify duplicates as codes were collated from the three researchers. Same coloured codes were synthesised to develop a general pattern of meaning, which we referred to as candidate themes. The sorting and collation approach would bring together all codes under each theme which then would facilitate further analysis and review (Bree et al., 2014 ).

The research team went on to review the relationship of the data and the codes that informed the themes. This is supported by Braun and Clarke ( 2012 , 2021 ) who posited that researchers should conduct a recursive review of the candidate themes in relation to the coded data items and the entire dataset. During the review, whenever themes were integrated or codes were moved to another theme, a new spreadsheet was created so that if further review was necessary, the old data and layout would still be available. Importantly, if the codes form a coherent and meaningful pattern, the theme makes a logical argument and may be representative of the data (Nowell et al., 2017 ). Furthermore, the team also reviewed the themes in relation to the data. This is because Nowell and others posited that themes should provide the most accurate interpretation of the data. The research team also discussed and wrote detailed analysis for each candidate theme identifying the main story behind each theme and how each one fit into the overall story about the data through the lens of the research questions. Finally, the researchers also linked quotes to final themes reached during the analysis. Illustrating findings with direct quotations from the participants strengthen the face validity and credibility of the research (Bryne, 2022 ; Patton, 2002 ; Nowell et al., 2017 ).

Student artefacts

The students’ artefacts (procedural flowcharts) in Figs.  5 , 6 and 7 were analysed using content analysis. Content analysis can be used to analyse written, verbal or visual representations (Cole, 1988 ; Elo & Kyngäs, 2008 ). Content analysis is ideal when there is a greater need to identify critical processes (Lederman, 1991 ). Unlike interviews, documents that are ideal for qualitative analysis should be developed independently without the researcher’s involvement (Merriam & Tisdell, 2015 ). In fact, the documents should not have been prepared for the purpose of research (Hughes & Goodwin, 2014 ), hence they are a stable and discrete data source (De Massis & Kotlar, 2014 ; Merriam & Tisdell, 2015 ). The students’ artefacts used in this study were not prepared for the purpose of the study but as a mathematics task. Deductive content analysis is used when the structure of analysis is implemented on the basis of previous knowledge and the purpose of the study is model testing or confirmation (Burns & Grove, 2009 ). Similarly, it is an analytical method that aims to test existing concepts, models or hypotheses in a new context (Kyngäs et al., 2020 ). They went further to note that researchers can use deductive analysis to determine how a model fit a new context.

Deductive content analysis follows three main stages: preparation, organising and reporting (Elo et al., 2014 ; Elo & Kyngäs, 2008 ). Firstly, preparation involves identifying the unit of analysis (Guthrie et al., 2004 ). In this study, the unit of analysis are the artefacts developed by the students. Furthermore, the phase requires the researcher to be immersed in the data reading and digesting to make sense of the whole set of data through reflexivity, open-mindedness and following the rationale of what guided participants’ narratives or in developing the artefact (Azungah, 2018 ). Secondly, a categorisation matrix based on existing knowledge should be developed or identified to facilitate the coding of the data according to categories (Hsieh & Shannon, 2005 ) (Table  1 ). Importantly, when using deductive content analysis, researchers require a theoretical structure or model from which they can build an analysis matrix (Kyngäs et al., 2020 ). Finally, the analysis results should be reported in ways that promote interpretation of the data and the results, for example, in tabular form (Elo & Kyngäs, 2008 ) (Fig.  2 ).

Stages followed during analysis of procedural flowcharts

The students’ procedural flowcharts were coded and interpreted on how they respond to different stages of problem solving. The researcher’s codes, interpretations and findings should be clearly derived and justified using the available data and then inform conclusions and interpretations for confirmability (Tobin & Begley, 2004 ). The artefacts were shared between the researcher and his supervisors; the analysis was done independently then reviewed by the researcher and his supervisors. Schreier ( 2012 ) recommended that analysis should be done by more than one person to promote thoroughness and broaden the interpretation of the data. Schreier went further to note that if the categorisation matrix is clear and of high quality, the coding should produce very little discrepancies. Very little discrepancies were observed except that some stages on the students’ procedural flowcharts overlapped between skills.

This section presents results from the analysis of the interviews data and student artefacts.

Semi-structured interviews

The thematic analysis of interviews resulted in two themes:

The utility of procedural flowcharts in supporting mathematics problem solving.

The utility of procedural flowcharts in supporting the integration of the four stages of mathematics problem solving.

In phase three, which prompted the targeted phase four study described in this study, teachers were asked the question, “How have you used procedural flowcharts to enhance teaching and learning of mathematics?” The question was not specific to problem solving but the teachers’ observations and perceptions strongly related to problem-solving and student-centred learning.

Theme 1 The utility of procedural flowcharts generally supports mathematics problem solving

The visual nature of procedural flowcharts was seen as an advantage to both teachers and students. For students, drawing a flowchart was easier than writing paragraphs to explain how they had arrived at the intended solution. For teachers, the flowchart was easier to process for timely feedback to students. Developing a procedural flowchart at the first checkpoint in the PSMT allows teachers to provide valuable feedback as the procedural flowchart can be used to represent several processes compared to written because of its visual nature. Engagement can be promoted because students can use the targeted feedback to improve their solutions as they will have provided a detailed overview of how they propose to solve the problem.

They present steps in diagrammatic form which is easy to process and easy to understand and process… students prefer them more as its in diagrammatic form and I have witnessed more students engaging. (Participant 8, phase three study) I find it (visual) a really efficient way for me to look at the proposed individual students processes and provide relevant feedback to the student or for the student to consider. And, you know, once the students are comfortable with using these procedural flowcharts you know, I find it much easier for me to give them relevant feedback, and I actually find that feedback more worthwhile than feedback we used to give them, you know, that was just based on what they wrote in paragraphs,…students get to practice in creating their own visual display, which communicates their intended strategies to solve the problem, then they have opportunities to use it, and fine tune it as they work out the problem … student developed procedural flow charts, they represent a student’s maths knowledge in a visual way. (Ms. Simon).

Identifying students’ competencies early was seen as central to successful problem solving as it provided opportunities for early intervention. Results showed that teachers viewed procedural flowcharts as a resource that could be used to identify gaps in skills, level of understanding and misconceptions that could affect successful and meaningful execution of a problem-solving task. Going through a student-developed flowchart during problem solving provided the teachers with insight into the student’s level of understanding of the problem and how the effectiveness of the procedures proposed to address the problem. This is critical for tasks that require students to develop a report detailing the solution at the end of developing the solution. Teachers can get the opportunity to gain an insight of the proposed solution before the student commit to write the report. The procedural flowchart provides the bigger picture of the solution plan which might expose gaps in knowledge.

I found it quite useful because I can identify what kids or which kids are competent in what, which sort of problem-solving skills. And I can identify misconceptions that students have or gaps in students understanding. (Participant 1, phase three study) It also to me highlights gaps in students’ knowledge in unique ways that students intend to reach a solution because the use of the procedural flow chart encourages students to explain the steps or procedures behind any mathematical manipulation that you know they're intending to use. And it's something that was much more difficult to determine prior to using procedural flow charts… I've also used you know, student developed procedural flow charts to ascertain how narrow or wide the students’ knowledge is and that's also something that wasn't obvious to make a judgement about prior to using procedural flow charts. (Ms. Simon)

Problem solving was seen as student-centred. If procedural flowcharts could be used to support problem solving, then they could facilitate an environment where students were the ones to do most of the work. The students could develop procedural flowcharts showing how they will solve a PSMT task using concepts and procedures they have learnt. The open-ended nature of the problem in a PSMT provides opportunities for diverse solutions that are validated through mathematical justifications. The visual nature of procedural flowcharts makes them more efficient to navigate compared to text.

Mathematics goes from being very dry and dusty to being something which is actually creative and interesting and evolving, starting to get kids actually engaging and having to back themselves. (Participant 7, phase three study) As a teacher, I find that procedural flowcharts are a really efficient way to ascertain the ways that students have considered and how they are going to solve a problem … It engages the students from start to finish, you know in different ways this method demands students to compare, interpret, analyse, reason, evaluate, and to an extent justify as they develop this solution. (Ms. Simon)

Similarly, results showed that procedural flowcharts could be used as a resource to promote collaborative learning and scaffolding. Students could be asked to collaboratively develop a procedural flowchart or could be provided with one to follow as they worked towards solving the problem. Collaborative development of procedural flowcharts can support problem solving as students can bring their different mathematical understanding to develop a solution from different perspectives.

Sometimes, you know, I get students to work on it in groups as they share ideas and get that mathematisation happening. So, it's really helpful there … I looked at the PSMT and its Marking Guide, and develop a more detailed procedural flowchart for students to use as a scaffold to guide them through the process. So, procedural flowcharts provide a structure in a more visual way for students to know what to do next. (Ms. Simon)

Ms. Simon shared her detailed procedural flowchart in Fig.  3 that she used to guide students in PSMTs.

Ms. Simon’s procedural flowchart on problem solving

The participants also observed that procedural flowcharts could be used to promote opportunities for solution evaluation which played an important role in problem solving. Loops can be introduced in procedural flowcharts to provide opportunities for reflection and reasoning as alternative paths provide flexibility while the solution is being developed. Following Fig.  4 are participants’ comments referring to the figure which was among procedural flowcharts shared with participants as examples of how they can be used to teach syllabus identified Mathematical Methods concepts. The Mathematical Methods syllabus expects students to “recognise the distinction between functions and relations and use the vertical line test to determine whether a relation is a function” (QCAA, 2018 p. 20).

The cycle approach, the feeding back in the feeding back out that type of stuff, you know, that is when we starting to teach students how to think . (Participant 7, phase three study) Complex procedural flowcharts like the one you provided guide students in making key decisions as they work through solutions which is key to critical thinking and judgement and these two are very important in maths. (Participant 8, phase three study) I also sincerely believe that procedural flowcharts are a way to get students to develop and demonstrate the critical thinking skills, which PSMTs are designed to assess. Students inadvertently have to use their critical thinking skills to analyse and reason as they search for different ways to obtain a solution to the problem presented in the PSMT … the use of procedural flowcharts naturally permits students to develop their critical thinking skills as it gets their brain into a problem-solving mode as they go through higher order thinking skills such as analysis, reasoning and synthesis and the like … this visual way of presenting solution provides students with opportunities to think differently, which they're not used to do, and it leads them to reflect and compare. (Ms. Simon)

Procedural flowchart on distinguishing functions and relations

Problem solving of non-routine problems uses a structure that should be followed. Resources that are intended to support problem solving in students can be used to support the integration of the stages involved in problem solving.

Theme 2 The utility of procedural flowcharts in supporting the integration of the four stages of mathematics problem solving.

Procedural flowcharts can support the flow of ideas and processes in the four stages during problem-solving and modelling task in Mathematical Methods subject. Literature synthesis in this study identified the four stages as:

Identification of problem and mathematics strategies than can solve the problem.

Implementation.

Evaluation and justification.

Communicating the solution.

Similarly, QCAA flowchart on PSMT identifies the four stages as formulate, solve, evaluate and verify, and communicate.

The logical sequencing of the stages of mathematics problem solving is crucial to solving and communicating the solution to the problem. Development of procedural flowcharts can play an important role in problem solving through fostering the logical sequencing of processes to reach a solution. Participants noted that the development of procedural flowcharts provides opportunities for showing the flow of ideas and processes which lay out an overview of how different stages connect into a bigger framework of the solution. Furthermore, it can help show how different pieces of a puzzle interconnect, in this case how all the components of the solution interconnect and develop to address the problem. In fact, procedural flowcharts can be used to show how the different mathematics concepts students have learnt can be brought together in a logical way to respond to a problem.

Procedural flowcharts help students sum up and connect the pieces together… connect the bits of knowledge together. (Participant 4, phase three study) Really good how it organises the steps and explains where you need to go if you're at a certain part in a procedure. (Participant 2, phase three study) Potentially, it's also an excellent visual presentation, which shows a student's draft of their logical sequence of processes that they're intending to develop to solve the problem … So, the steps students need to follow actually flows logically. So really given a real-life scenario they need to solve in a PSMT students need to mathematise it and turn it into a math plan, where they execute their process, evaluate and verify it and then conclude … so we use procedural flowcharts to reinforce the structure of how to approach problem-solving … kids, you know, they really struggling, you know, presenting things in a logical way, because they presume that we know what they're thinking . (Ms. Simon)

Developing procedural flowcharts provided students with opportunities to plan the solution informed by the stages of problem solving. Teachers could reinforce the structure of problem solving by telling students what they could expect to be included on the procedural flowchart. Procedural flowcharts can be used as a visual tool to highlight all the critical stages that are included during the planning of the solution.

I tell the students, “I need to see how you have interpreted the problem that you need to solve. I need to see how you formulated your model that involves the process of mathematisation, where you move from the real world into the maths world, and I need to see all the different skills you're intending to use to arrive at your solution.” (Ms. Simon)

Similarly, procedural flowcharts could visually represent more than one strategy in the “identify and execute mathematics procedures that can solve the problem” stage, thereby providing a critical resource to demonstrate flexibility. When there are multiple ways of addressing a problem, developing a procedural flowchart can provide an opportunity of showing all possible paths or relationships between different paths to the solution, thus promoting flexibility. Procedural flowcharts provide an opportunity to show how different procedures can be used or integrated to solve a problem.

Students are expected to show evidence that they have the knowledge of solving the problem using several ways to get to the same solution. So, it goes beyond the students’ preferred way of answering a question and actually highlights the importance of flexibility when it comes to processes and strategies of solving a problem … By using procedural flowcharts, I'm saying to the students, “Apart from your preferred way of solving the problem, give me a map of other routes, you can also use to get to your destination.” (Ms. Simon)

The results also indicated that procedural flowcharts could be used to identify strengths and limitations of procedures in the “evaluate solution” stage and thus demonstrate the reasonableness of the answer. Having more than one way of solving a problem on a procedural flowchart helps in comparing and evaluating the most ideal way to address the problem.

And I'm finding that, you know, as students go through, and they compare the different processes, you know, the strengths and limitations, literally stare them in the face. So, they don't have to. They're not ... they don't struggle as much as they used to in coming up with those sorts of answers … it's also a really easy way that once the students reach the next phase, which is the evaluating verified stage, they can go back to their procedural flow chart and identify and explain strengths and limitations of their model … It's a convenient way for students to show their reasonableness of their solution by comparing strengths and weaknesses of all the strategies presented on the procedural flowchart, something that they've struggled with in the past. (Ms. Simon)

The results from the interview show that the procedural flowcharts supported efficient communication of the steps to be followed in developing the solution to the problem. Student-developed procedural flowcharts allowed the teacher to have an insight and overview of the solution to the problem earlier in the assessment task. In addition, they provided an alternative way of presenting their solution to the teacher.

I expect students to use the procedural flowchart as a way to communicate to me how they're planning to solve the scenario in the PSMT…It's also one of the parts that students are expected to hand in to me on one of the check points, and I find it a really efficient way for me to look at, you know, a proposed individual students processes, and provide relevant feedback to the student to consider in a really efficient way…I just found that it helps students communicate their solution to a problem in lots of different ways that challenges students to logically present a solution. (Ms. Simon)

She went on to say,

Students also found it challenging to communicate their ideas in one or two paragraphs, when more than one process or step was required to solve the problem. So, I found that, you know, procedural flowcharts, have filled this gap really nicely, as that provides students with a simple tool that they can use to present a visual overview of the processes they've chosen to use to solve the problem. And so, for me, as a teacher, procedural flowcharts are an efficient way for me to scan the intended processes that an individual student is proposing to use to solve the problem in their authentic way and provide them with valuable feedback.

In summary, the teacher’s experiences, views and perceptions showed that procedural flowcharts can be a valuable resource in supporting students in all four stages of problem solving.

Students’ artefacts

The student-generated flowcharts in this part of the research gave an insight into students’ understanding as they planned how to solve the problem presented to them. Students were expected to use the problem-solving stages to successfully develop solutions to problems. Their de-identified procedural flowcharts are shown in Figs.  5 , 6 and 7 .

Procedural flowchart developed by student 1

Procedural flowchart developed by student 2

Collaboratively developed procedural flowchart

Students 1 and 2 also collaboratively developed a procedural flowchart, shown as Fig.  7 .

This discussion is presented as two sections: (1) how developing procedural flowcharts can support mathematics problem solving and (2) how developing procedural flowcharts support the integration of the different stages of mathematics problem solving. This study although limited by sample size highlighted how developing procedural flowcharts can support mathematics problem solving, can reinforce the structure of the solution to a problem and can help develop metacognitive skills among students. The different stages involved in problem solving inform the process of developing the solution to the problem. The focus on problem-based learning has signified the need to introduce resources that can support students and teachers in developing and structuring solutions to problems. Results from this study have also provided discussion points on how procedural flowcharts can have a positive impact in mathematics problem solving.

Procedural flowcharts can support mathematics problem solving

Procedural flowcharts help in visualising the process of problem solving. The results described in this study show that student-generated flowcharts can provide an overview of the proposed solution to the problem. The study noted that students preferred developing procedural flowcharts rather than writing how they planned to find a solution to the problem. The teachers also preferred visual aids because they were easier and quicker to process and facilitated understanding of the steps taken to reach the solution. These results are consistent with the findings of other researchers (McGowan & Boscia, 2016 ; Raiyn, 2016 ). The results are also consistent with Grosskinsky and colleagues’ ( 2019 ) findings that flowcharts break complex information into different tasks and show how they are connected, thereby enhancing understanding of the process. Consequently, they allow teachers to provide timely feedback at a checkpoint compared to the time a teacher would take to go through a written draft. Procedural flowcharts connect procedures and processes in a solution to the problem (Chinofunga et al., 2022 ). Thus, the feedback provided by the teacher can be more targeted to a particular stage identified on the procedural flowchart, making the feedback more effective and worthwhile. The development of a procedural flowchart during problem solving can be viewed as a visual representation of students’ plan and understanding of how they plan to solve the problem as demonstrated in Figs.  5 , 6 and 7 .

In this study, Ms. Simon noted that procedural flowcharts can represented students’ knowledge or thinking in a visual form, which is consistent with Owens and Clements’ ( 1998 ) findings that visual representations are cognitive constructs. Consequently, they can facilitate evaluation of such knowledge. This study noted that developing procedural flowcharts can provide opportunities to identify gaps in students’ understanding and problem-solving skills. It also noted that providing students with opportunities to develop procedural flowcharts may expose students’ misconceptions, the depth and breadth of their understanding of the problem and how they plan to solve the problem. This is supported by significant research (Grosskinsky et al., 2019 ; Norton et al., 2007 ; Vale & Barbosa, 2018 ), which identified flowcharts as a resource in helping visualise and recognise students’ understanding of a problem and communication of the solution. Thus, providing teachers with opportunities to have an insight into students’ thinking can facilitate intervention early in the process. The results in this study showed that when students develop their own plan on how to respond to a problem, they are at the centre of their learning. However, scaffolding and collaborative learning can also support problem solving.

Vygotsky ( 1978 ) posited that in the Zone of Proximal Development, collaborative learning and scaffolding can facilitate understanding. In this study, the results indicated that a teacher-developed procedural flowchart can be used to guide students in developing a solution to a problem. These results are consistent with Davidowitz and Rollnick’s study that concluded that flowcharts provide a bigger picture of how to solve the problem. In Queensland, the QCAA has developed a flowchart (see Appendix 1 ) to guide schools on problem-solving and modelling tasks. It highlights the significant stages to be considered during the process and how they relate to each other. Teachers are encouraged to contextualise official documents to suit their school and classes. In such cases, a procedural flowchart acts as a scaffolding resource in directing students on how to develop the solution to the problem. The findings are consistent with previous literature that flowcharts can give an overall direction of the process, help explain what is involved, may help reduce cognitive load and allow students to focus on complex tasks (Davidowitz & Rollnick, 2001 ; Norton et al., 2007 ; Sweller et al., 2019 ).

In addition to being a scaffolding resource, results showed that procedural flowcharts can be developed collaboratively providing students with an opportunity to share their solution to the problem. Being a scaffolding resource or a resource to use in a community of learning highlights the importance of procedural flowcharts in promoting learning within a zone of proximal development, as posited by Davidowitz and Rollnick ( 2001 ). Scaffolding students to problem solve and develop procedural flowcharts collaboratively provides students with the opportunity to be at the centre of problem solving.

Research has identified problem solving as student-centred learning (Ahmad et al., 2010 ; Karp & Wasserman, 2015 ; Reinholz, 2020 ; Vale & Barbosa, 2018 ). The process of developing the procedural flowcharts as students plan for the solution provides students with opportunities to engage more with the problem. Results showed that when students developed procedural flowcharts themselves, mathematics learning transformed from students just being told what to do or follow procedures into something creative and interesting. As students develop procedural flowcharts, they use concepts they have learnt to develop a solution to an unfamiliar problem (Matty, 2016 ), thus engaging with learning from the beginning of the process until they finalise the solution. The results indicated that developing procedural flowcharts promoted students’ ability to not only integrate different procedures to solve the problem but also determine how and when the conditions were ideal to address the problem, providing opportunities to justify and evaluate the procedures that were used.

Deeper understanding of mathematics and relationships between concepts plays an important role in problem solving, and the results from this study showed that different procedures can be integrated to develop a solution to a problem. The participants observed that developing procedural flowcharts could support the brainstorming ideas as they developed the flowchart, as ideas may interlink in a non-linear way. Moreover, students are expected at different stages to make key decisions about the direction they will need to take to reach the solution to the problem, as more than one strategy may be available. For example, student 1 planned on using only technology to develop the models while student 2 considered both technology and algebra. This showed that student 2 applied flexibility in using alternative methods, thus demonstrating a deeper understanding of the problem. Equally important, Ms. Simon observed that as students developed their procedural flowcharts while planning the steps to reach a solution, they were required to analyse, conceptualise, reason, analyse, synthesise and evaluate, which are important attributes of deeper understanding. Fostering deeper understanding of mathematics is the key goal of using problem solving (Kim et al., 2012 ; King, 1995 ; Moon, 2008 ; QCAA, 2018 ). The results are additionally consistent with findings from Owens and Clements ( 1998 ) and Roam ( 2009 ), who posited that visual aids foster reasoning and show cognitive constructs. Similarly, logical sequencing of procedures and ways to execute a strategy expected when developing procedural flowchart can support deeper understanding, as posited by Parvaneh and Duncan ( 2021 ). When developing procedural flowchart, students are required to link ideas that are related or feed into another, creating a web of knowledge. Students are also required to identify the ways in which a concept is applied as they develop a solution, and this requires deeper understanding of mathematics. Working collaboratively can also support deeper and broader understanding of mathematics.

The procedural flowchart that was developed collaboratively by the two students demonstrated some of the skills that they did not demonstrate in their individual procedural flowcharts. Like student 2, the collaboratively developed flowchart included use of technology and algebra to determine the models for the three different cups. The students considered both rate of change and area under a curve in the task analysis. Apart from planning to use rate at a point, average rate and definite integration, they added the trapezoidal rule. Both average rate and definite integration were to be applied within the same intervals, building the scope for comparison. The trapezoidal rule would also compare with integration. The complexity of the collaboratively developed procedural flowchart concurred with Rogoff and others ( 1984 ) and Stone ( 1998 ), who suggested that a community of learning can expand current skills to higher levels than individuals could achieve on their own. It seems the students used the feedback provided by the teacher on their individually developed procedural flowcharts as scaffolding to develop a much more complex procedural flowchart with competing procedures to address the problem. Their individually developed flowcharts might have acted as reference points, as their initial plans were still included in the collaboratively developed plan but with better clarity. This observation is consistent with Guk and Kellogg ( 2007 ), Kirova and Jamison ( 2018 ) and Ouyang and colleagues ( 2022 ), who noted that scaffolding involving peers, teacher and other resources enhances complex problem-solving tasks and transfer of skills.

Supporting the integration of the different stages of mathematics problem solving

When students develop procedural flowcharts, it supports the logical sequencing of ideas from different stages into a process that ends with a solution. Problem solving follows a proposed order and procedural flowcharts visually display decision and/or action sequences in a logical order (Krohn, 1983 ). They are used when a sequenced order of ideas is emphasised, such as in problem solving (Cantatore & Stevens, 2016 ). This study concurs with Krohn, Cantatore and Stevens, as the results showed that procedural flowcharts could be used to organise steps and ideas logically as students worked towards developing a solution. Students’ procedural flowcharts are expected to be developed through the following stages: problem identification, problem mathematisation, planning and execution and finally evaluation. Such a structure can be reinforced by teachers by sharing a generic problem-solving flowchart outlining the stages so that students can then develop a problem-specific version. Importantly, students’ artefacts in Figs.  5 , 6 and 7 provided evidence of how procedural flowcharts support the different stages of problem-solving stages to create a logical and sequential flow of the solution (see Appendix 1 ). Similarly, Ms. Simon noted that while her students had previously had problems in presenting the steps to their solution in a logical way, she witnessed a significant improvement after she asked them to develop procedural flowcharts first. Further, the results are consistent with Chinofunga et al.’s ( 2022 ) work that procedural flowcharts can support procedural flexibility, as they can accommodate more than one procedure in the “identify and execute mathematics procedures that can solve the problem” stage. Thus, stages that require one procedure or more than one procedure can all be accommodated in a single procedural flowchart. Evaluating the different procedures is also a key stage in problem solving.

As students develop the solution to the problem and identify ways to address the problem, they also have to evaluate the procedures, reflecting on the limitations and strengths of the solutions they offer. Ms. Simon observed that her students had previously struggled with identifying strengths and weaknesses of different procedures. However, she noted that procedural flowcharts gave students the opportunity to reflect and compare as they planned the solution. For example, students could have the opportunity to reflect and compare rate at a point, average rate and integration so they can evaluate which strategy can best address he problem. The artefacts identified the different procedures the students used in planning the solution, enabling them to evaluate the effectiveness of each strategy. Thus, enhancing students’ capacity to make decisions and identify the optimal strategy to solve a problem aligns with the work of McGowan and Boscia ( 2016 ). Similarly, Chinofunga and colleagues’ findings noted that developing procedural flowcharts can be effective in evaluating different procedures as they can accommodate several procedures. The different stages that need to be followed during problem solving and the way the solution to the problem is logically presented are central to how the final product is communicated.

In this study, procedural flowcharts were used to communicate the plan to reach the solution to a problem. The length of time given to students to work on their problem-solving tasks in Queensland is fairly long (4 weeks) and students may struggle to remember some key processes along the way. Developing procedural flowcharts to gain an overview of the solution to the problem and share it with the teacher at an early checkpoint is of significant importance. In this study, Ms. Simon expected her students to share their procedural flowcharts early in the process for her to give feedback, thus making the flowcharts a communication tool. The procedural flowcharts developed by the students in Figs.  5 , 6 and 7 show how students proposed solving the problem. This result lends further support to the NCTM ( 2000 ) findings that visual representations can help students communicate their thinking before applying those thoughts to solving a problem. Ms. Simon also noted that before introducing students to procedural flowcharts, they did not have an overall coherent structure to follow, which presented challenges when they wanted to communicate a plan that involved more than one strategy. However, the students’ artefacts were meaningful, clearly articulating how the solution to the problem was being developed, thus demonstrating that procedural flowcharts can provide the structure that supports the coherent and logical communication of the solution to the problem by both teachers and students (Norton et al., 2007 ). The visual nature of the students’ responses in the form of procedural flowcharts is key to communicating the proposed solution to the problem.

Visual representations are a favourable alternative to narrative communication. Procedural flowcharts can help teachers to check students’ work faster and provide critical feedback in a timely manner. Ms. Simon noted that the use of procedural flowcharts provided her with the opportunity to provide feedback faster and more effectively earlier in the task because the charts provided her with an overview of the whole proposed solution. Considering that students are expected to write a report of 2000 words or 10 pages on the task, the procedural flowchart provides the opportunity to present large amounts of information in just one visual representation. Raiyn ( 2016 ) noted that visual representations can be a quicker way to evaluate a solution and represent large amounts of information.

The procedural flowcharts that were created by students in this study demonstrate that they can be effective in supporting the development of problem-solving skills. This study suggests that including procedural flowcharts in problem solving may support teachers and students in communicating efficiently about how to solve the problem. For students, it is a resource that provides the solution overview, while teachers can consider it as a mental representation of students’ thinking as they plan the steps to reach a solution. Student-developed procedural flowcharts may represent how a student visualises a solution to a problem after brainstorming different pathways and different decision-making stages.

Moreover, as highlighted in this study, the visual nature of procedural flowcharts may offer a diverse range of support for problem solving. Procedural flowcharts make it easy to process and provide timely feedback that in turn might help students engage with the problem meaningfully. Furthermore, they may also provide a structure of the problem-solving process and guide students through the problem-solving process. Navigating through stages of problem solving might be supported by having students design procedural flowcharts first and then execute the plan. Indeed, this study showed that the ability of procedural flowcharts to represent multiple procedures, evaluation stages or loops and alternative paths helps students reflect and think about how to present a logically cohesive solution. Importantly, procedural flowcharts have also been identified as a resource that can help students communicate the solution to the problem. Procedural flowcharts have been noted to support deeper understanding as it may facilitate analysis, logical sequencing, reflection, reasoning, evaluation and communication. Although the in-depth study involved one teacher and three artefacts from her students, which is a very small sample to be conclusive, it identified the numerous advantages that procedural flowcharts bring to mathematics learning and teaching, particularly in terms of supporting the development of problem-solving skills. The study calls for further investigation on how procedural flowcharts can support students’ problem solving.

Ahmad, A., Tarmizi, R. A., & Nawawi, M. (2010). Visual representations in mathematical word problem-solving among form four students in malacca. Procedia - Social and Behavioral Sciences, 8 , 356–361. https://doi.org/10.1016/j.sbspro.2010.12.050

Article   Google Scholar  

Anderson, J. (2014). Forging new opportunities for problem solving in Australian mathematics classrooms through the first national mathematics curriculum. In Y. Li & G. Lappan (Eds.), Mathematics curriculum in school education (pp. 209–230). Springer.

Chapter   Google Scholar  

Arcavi, A. (2003). The role of visual representations in the learning of mathematics. Educational Studies in Mathematics, 52 (3), 215–241. https://doi.org/10.1023/A:1024312321077

Artigue, M., & Blomhøj, M. (2013). Conceptualizing Inquiry-Based Education in Mathematics. ZDM, 45 (6), 797–810. https://doi.org/10.1007/s11858-013-0506-6

Artigue, M., Bosch, M., Doorman, M., Juhász, P., Kvasz, L., & Maass, K. (2020). Inquiry based mathematics education and the development of learning trajectories. Teaching Mathematics and Computer Science, 18 (3), 63–89. https://doi.org/10.5485/TMCS.2020.0505

Australia Curriculum and Reporting Authority. (2014). Mathematics proficiencies (Version 8.4) . https://www.australiancurriculum.edu.au/resources/mathematics-proficiencies/portfolios/problem-solving/

Azungah, T. (2018). Qualitative research: deductive and inductive approaches to data analysis. Qualitative Research Journal, 18 (4), 383–400. https://doi.org/10.1108/QRJ-D-18-00035

Berger, M. (1998). Graphic calculators: An Interpretative framework. For the Learning of Mathematics, 18 (2), 13–20.

Google Scholar  

Blum, W., & Leiß, D. (2007). Deal with modelling problems. Mathematical Modelling: Education, Engineering and Economics, 12 , 222. https://doi.org/10.1533/9780857099419.5.221

Bos, R., & van den Bogaart, T. (2022). Heuristic trees as a digital tool to foster compression and decompression in problem-solving. Digital Experiences in Mathematics Education, 8 (2), 157–182. https://doi.org/10.1007/s40751-022-00101-6

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Braun, V., & Clarke, V. (2012). Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA Handbook of Research Methods in Psychology, Research Designs (Vol. 2, pp. 57–71). American Psychological Association.

Braun, V., & Clarke, V. (2021). One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology, 18 (3), 328–352. https://doi.org/10.1080/14780887.2020.1769238

Bree, R. T., Dunne, K., Brereton, B., Gallagher, G., & Dallat, J. (2014). Engaging learning and addressing over-assessment in the Science laboratory: Solving a pervasive problem. The All-Ireland Journal of Teaching and Learning in Higher Education, 6 (3), 206.1–206.36. http://ojs.aishe.org/index.php/aishe-j/article/viewFile/206/290

Burns, N., & Grove, S. (2009). The practice of nursing research: Appraisal, synthesis and generation of evidence (6th ed.). St. Louis: Saunders Elsevier. https://doi.org/10.7748/ns2013.04.27.31.30.b1488

Book   Google Scholar  

Byrne, D. (2022). A worked example of Braun and Clarke’s approach to reflexive thematic analysis. Quality & Quantity, 56 (3), 1391–1412. https://doi.org/10.1007/s11135-021-01182-y

Cai, J. (2010). Helping elementary school students become successful mathematical problem solvers. In D. Lambdin (Ed.), Teaching and learning mathematics: Translating research to the elementary classroom (pp. 9–14). Reston, VA: National Council of Teachers of Mathematics.

Cantatore, F., & Stevens, I. (2016). Making connections : Incorporating visual learning in law subjects through mind mapping and flowcharts. Canterbury Law Review, 22 (1), 153–170. https://doi.org/10.3316/agis_archive.20173661

Chinofunga, M. D., Chigeza, P., & Taylor, S. (2022). Procedural flowcharts can enhance senior secondary mathematics. In N. Fitzallen, C. Murphy, & V. Hatisaru (Eds.), Mathematical confluences and journeys (Proceedings of the 44th Annual Conference of the Mathematics Education Research Group of Australasia, July 3-7) (pp. 130–137). Launceston: MERGA. https://files.eric.ed.gov/fulltext/ED623874.pdf

Colburn, A. (2000). An inquiry primer. Science Scope, 23 (6), 42–44. http://www.cyberbee.com/inquiryprimer.pdf

Cole, F. L. (1988). Content analysis: Process and application. Clinical Nurse Specialist, 2 (1), 53–57. https://doi.org/10.1097/00002800-198800210-00025

Article   CAS   PubMed   Google Scholar  

Csíkos, C., Szitányi, J., & Kelemen, R. (2012). The effects of using drawings in developing young children’s mathematical word problem solving: A design experiment with third-grade Hungarian students. Educational Studies in Mathematics, 81 , 47–65. https://doi.org/10.1007/s10649-011-9360-z

Davidowitz, B., & Rollnick, M. (2001). Effectiveness of flow diagrams as a strategy for learning in laboratories. Australian Journal of Education in Chemistry, (57), 18–24. https://search.informit.org/doi/10.3316/aeipt.129151

De Massis, A., & Kotlar, J. (2014). The case study method in family business research: Guidelines for qualitative scholarship. Journal of Family Business Strategy, 5 (1), 15–29. https://doi.org/10.1016/j.jfbs.2014.01.007

Dorier, J.-L., & Maass, K. (2020). Inquiry-based mathematics education. In S. Lerman (Ed.), Encyclopedia of mathematics education (pp. 384–388). Springer. https://doi.org/10.1007/978-3-030-15789-0_176

Doyle, K. M. (2005). Mathematical problem solving: A need for literacy. In F. Bryer, B. Bartlett, & D. Roebuck (Eds.), Proceedings Stimulating the “Action” as participants in participatory research 2 (pp. 39–45). Australia: Surfers Paradise.

Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62 (1), 107–115. https://doi.org/10.1111/j.1365-2648.2007.04569.x

Article   PubMed   Google Scholar  

Elo, S., Kääriäinen, M., Kanste, O., Pölkki, T., Utriainen, K., & Kyngäs, H. (2014). Qualitative content analysis: A focus on trustworthiness. SAGE Open, 4 (1), 215824401452263. https://doi.org/10.1177/2158244014522633

English, L., & Gainsburg, J. (2016). Problem solving in a 21st-century mathematics curriculum. In L. English & D. Kirshner (Eds.), Handbook of international research in mathematics education (3rd ed., pp. 313–335). New York, NY: Routledge.

Galbraith, P., & Stillman, G. (2006). A framework for identifying student blockages during transitions in the modelling process. ZDM – Mathematics Education, 38 (2), 143–162. https://doi.org/10.1007/BF02655886

Geiger, V., Galbraith, P., Niss, M., & Delzoppo, C. (2021). Developing a task design and implementation framework for fostering mathematical modelling competencies. Educational Studies in Mathematics, 109 (2), 313–336. https://doi.org/10.1007/s10649-021-10039-y

Gencer, S. (2023). Development and use of flowchart for preservice chemistry teachers’ problem solving on the first law of thermodynamics. Journal of Chemical Education, 100 (9), 3393–3401. https://doi.org/10.1021/acs.jchemed.3c00224

Article   CAS   ADS   Google Scholar  

Grosskinsky, D. K., Jørgensen, K., & Hammer úr Skúoy, K. (2019). A flowchart as a tool to support student learning in a laboratory exercise. Dansk Universitetspædagogisk Tidsskrift, 14 (26), 23–35. https://doi.org/10.7146/dut.v14i26.104402

Guion, L. A., Diehl, D. C., & McDonald, D. (2011). Triangulation: Establishing the validity of qualitative studies. EDIS, (8), 3–3. https://doi.org/10.32473/edis-fy394-2011

Guk, I., & Kellogg, D. (2007). The ZPD and whole class teaching: Teacher-led and student-led interactional mediation of tasks. Language Teaching Research, 11 (3), 281–299. https://doi.org/10.1177/1362168807077561

Guthrie, J., Petty, R., Yongvanich, K., & Ricceri, F. (2004). Using content analysis as a research method to inquire into intellectual capital reporting. Journal of Intellectual Capital, 5 (2), 282–293. https://doi.org/10.1108/14691930410533704

Hacker, D. J., Dunlosky, J., & Graesser, A. C. (Eds.). (1998). Metacognition in educational theory and practice (1st ed.). Routledge. https://doi.org/10.4324/9781410602350

Hadamard, J. (1945). The psychology of invention in the mathematical field . Princeton, NJ: Princeton University Press.

Hankeln, C. (2020). Mathematical modeling in Germany and France: A comparison of students’ modeling processes. Educational Studies in Mathematics, 103 (2), 209–229. https://doi.org/10.1007/s10649-019-09931-5

Hooshyar, D., Ahmad, R. B., Yousefi, M., Fathi, M., Horng, S.-J., & Lim, H. (2016). Applying an online game-based formative assessment in a flowchart-based intelligent tutoring system for improving problem-solving skills. Computers and Education, 94 , 18–36. https://doi.org/10.1016/j.compedu.2015.10.013

Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15 (9), 1277–1288. https://doi.org/10.1177/1049732305276687

Huang, X., Huang, R., & Trouche, L. (2023). Teachers’ learning from addressing the challenges of online teaching in a time of pandemic: A case in Shanghai. Educational Studies in Mathematics, 112 (1), 103–121. https://doi.org/10.1007/s10649-022-10172-2

Hughes, J. R. A., & Goodwin, J. (2014). Editors’ introduction: Human documents and archival research . University of Leicester. Chapter. https://hdl.handle.net/2381/31547

Jitendra, A. K., Dupuis, D. N., Rodriguez, M. C., Zaslofsky, A. F., Slater, S., Cozine-Corroy, K., & Church, C. (2013). A randomized controlled trial of the impact of schema-based instruction on mathematical outcomes for third-grade students with mathematics difficulties. The Elementary School Journal, 114 (2), 252–276. https://doi.org/10.1086/673199

Jonassen, D. H. (2012). Designing for decision making. Educational Technology Research and Development, 60 (2), 341–359. https://doi.org/10.1007/s11423-011-9230-5

Kaitera, S., & Harmoinen, S. (2022). Developing mathematical problem-solving skills in primary school by using visual representations on heuristics. LUMAT: International Journal on Math, Science and Technology Education, 10 (2), 111–146. https://doi.org/10.31129/LUMAT.10.2.1696

Karp, A., & Wasserman, N. (2015). Mathematics in middle and secondary school: A problem-solving approach . Charlotte, North Carolina: Information Age Publishing Inc.

Kim, K., Sharma, P., Land, S. M., & Furlong, K. P. (2012). Effects of active learning on enhancing student critical thinking in an undergraduate general science course. Innovative Higher Education, 38 (3), 223–235. https://doi.org/10.1007/s10755-012-9236-x

King, A. (1995). Designing the instructional process to enhance critical thinking across the curriculum: Inquiring minds really do want to know: Using questioning to teach critical thinking. Teaching of Psychology, 22 (1), 13–17. https://doi.org/10.1207/s15328023top2201_5

Article   MathSciNet   Google Scholar  

King, N. (2004). Using templates in the thematic analysis of text. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 257–270). London, UK: Sage. https://doi.org/10.4135/9781446280119

Kingsdorf, S., & Krawec, J. (2014). Error analysis of mathematical word problem solving across students with and without learning disabilities. Learning Disabilities Research & Practice, 29 (2), 66–74. https://doi.org/10.1111/ldrp.12029

Kirova, A., & Jamison, N. M. (2018). Peer scaffolding techniques and approaches in preschool children’s multiliteracy practices with iPads. Journal of Early Childhood Research, 16 (3), 245–257. https://doi.org/10.1177/1476718X18775762

Koellner, K., Jacobs, J., & Borko, H. (2011). Mathematics professional development: Critical features for developing leadership skills and building teachers’ capacity. Mathematics Teacher Education & Development, 13 (1), 115–136. Retrieved from https://files.eric.ed.gov/fulltext/EJ960952.pdf

Krawec, J. L. (2014). Problem representation and mathematical problem solving of students of varying math ability. Journal of Learning Disabilities, 47 , 103–115. https://doi.org/10.1177/0022219412436976

Krohn, G. S. (1983). Flowcharts used for procedural instructions. Human Factors, 25 (5), 573–581. https://doi.org/10.1177/001872088302500511

Krutetskii, V. A. (1976). The psychology of mathematical abilities in schoolchildren . Chicago: University of Chicago Press.

Kules, B. (2016). Computational thinking is critical thinking: Connecting to university discourse, goals, and learning outcomes. Proceedings of the Association for Information Science and Technology, 53 (1), 1–6. https://doi.org/10.1002/pra2.2016.14505301092

Kyngäs, H., Mikkonen, K., & Kääriäinen, M. (2020). The application of content analysis in nursing science research . Cham: Springer. https://doi.org/10.1007/978-3-030-30199-6

Laughlin, P. R., Hatch, E. C., Silver, J. S., & Boh, L. (2006). Groups perform better than the best individuals on letters-to-numbers problems: Effects of group size. Journal of Personality and Social Psychology, 90 (4), 644–651. https://doi.org/10.1037/0022-3514.90.4.644

Lederman, R. P. (1991). Content analysis: Steps to a more precise coding procedure. MCN, The American Journal of Maternal Child Nursing, 16 (5), 275–275. https://doi.org/10.1097/00005721-199109000-00012

Ledin, P., & Machin, D. (2020). The misleading nature of flow charts and diagrams in organizational communication: The case of performance management of preschools in Sweden. Semiotica, 2020 (236), 405–425. https://doi.org/10.1515/sem-2020-0032

Lester, F. (2013). Thoughts about research on mathematical problem-solving instruction. The Mathematics Enthusiast, 10 (1–2), 245–278.

Lester, F. K., & Cai, J. (2016). Can mathematical problem solving be taught? Preliminary answers from 30 years of research. In P. Felmer, E. Pehkonen, & J. Kilpatrick (Eds.), Posing and solving mathematical problems: Advances and new perspectives (pp. 117–135). Springer International Publishing. https://doi.org/10.1007/978-3-319-28023-3_8

Lohse, G., Biolsi, K., Walker, N., & Rueter, H. (1994). A classification of visual representations. Communications of the ACM, 37 (12), 36–49. https://doi.org/10.1145/198366.198376

Makar, K. (2012). The pedagogy of mathematical inquiry. In R. Gillies (Ed.), Pedagogy: New developments in the learning sciences (pp. 371–397). Hauppauge, N.Y.: Nova Science Publishers.

Mason, J. (2016). When is a problem…? “When” is actually the problem! In P. Felmer, E. Pehkonen, & J. Kilpatrick (Eds.), Posing and solving mathematical problems. Advances and new perspectives (pp. 263–287). Switzerland: Springer. https://doi.org/10.1007/978-3-319-28023-3_16

Matty, A. N. (2016). A study on how inquiry based instruction impacts student achievement in mathematics at the high school level . ProQuest Dissertations Publishing. https://www.proquest.com/openview/da895b80797c90f9382f0c9a948f7f68/1?pq-origsite=gscholar&cbl=18750

McCormick, N. J., Clark, L. M., & Raines, J. M. (2015). Engaging students in critical thinking and problem-solving: A brief review of the literature. Journal of Studies in Education, 5 (4), 100–113. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.960.810&rep=rep1&type=pdf

McGowan, M. M., & Boscia, M. W. (2016). Opening more than just a bag: Unlocking the flowchart as an effective problem-solving tool. The Journal of Health Administration Education, 33 (1), 211–219.

Merriam, S. B., & Tisdell, E. J. (2015). Qualitative research: A guide to design and implementation (4th ed.). Newark: Wiley.

Moon, J. (2008). Critical thinking: An exploration of theory and practice . Routledge. https://doi.org/10.4324/9780203944882

National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics . Reston, VA: National Council of Teachers of Mathematics.

Nieuwoudt, S. (2015). Developing a model for problem-solving in a Grade 4 mathematics classroom. Pythagoras, 36 (2), 1–7. https://doi.org/10.4102/pythagoras.v36i2.275

Norton, S. J., McRobbie, C. J., & Ginns, I. S. (2007). Problem-solving in a middle school robotics design classroom. Research in Science Education, 37 (3), 261–277. https://doi.org/10.1007/s11165-006-9025-6

Article   ADS   Google Scholar  

Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16 (1), 1609406917733847. https://doi.org/10.1177/1609406917733847

Ouyang, F., Chen, S., Yang, Y., & Chen, Y. (2022). Examining the effects of three group-level metacognitive scaffoldings on in-service teachers’ knowledge building. Journal of Educational Computing Research, 60 (2), 352–379. https://doi.org/10.1177/07356331211030847

Owens, K. D., & Clements, M. A. (1998). Representations in spatial problem-solving in the classroom. The Journal of Mathematical Behavior, 17 (2), 197–218. https://doi.org/10.1016/S0364-0213(99)80059-7

Ozuem, W., Willis, M., & Howell, K. (2022). Thematic analysis without paradox: Sensemaking and context. Qualitative Market Research, 25 (1), 143–157. https://doi.org/10.1108/QMR-07-2021-0092

Pape, S. J., & Tchoshanov, M. A. (2001). The role of representation(s) in developing mathematical understanding. Theory into Practice, 40 (2), 118–127. https://doi.org/10.1207/s15430421tip4002_6

Parvaneh, H., & Duncan, G. J. (2021). The role of robotics in the development of creativity, critical thinking and algorithmic thinking. Australian Primary Mathematics Classroom, 26 (3), 9–13. https://doi.org/10.3316/informit.448545849534966

Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Sage Publications.

Polya, G. (1945). How to solve it: A new aspect of mathematical method . Princeton, NJ: Princeton University Press.

Polya, G. (1957). How to solve it: A new aspect of mathematical method . Princeton: Princeton University Press.

Polya, G. (1971). How to solve it: A new aspect of mathematical method (2nd ed.). Princeton University Press.

Queensland Curriculum and Assessment Authority (QCAA). (2018). Mathematical methods. general senior syllabus . Brisbane: Queensland Curriculum and Assessment Authority. https://www.qcaa.qld.edu.au/downloads/senior-qce/syllabuses/snr_maths_methods_19_syll.pdf

Raiyn, J. (2016). The role of visual learning in improving students’ high-order thinking skills. Journal of Education and Practice, 7 , 115–121. https://www.learntechlib.org/p/195092/

Reingewertz, Y. (2013). Teaching macroeconomics through flowcharts. International Review of Economics Education, 14 , 86–93. https://doi.org/10.1016/j.iree.2013.10.004

Reinholz, D. L. (2020). Five practices for supporting inquiry in analysis. Problems Resources and Issues in Mathematics Undergraduate Studies, 30 (1), 19–35. https://doi.org/10.1080/10511970.2018.1500955

Roam, D. (2009). The back of the napkin: Solving problems and selling ideas with pictures (1st ed.). Singapore: Marshall Cavendish International (Asia) Private Limited.

Rogoff, B., Malkin, C., & Gilbride, K. (1984). Interaction with babies as guidance in development. New Directions for Child and Adolescent Development, 1984 (23), 31–44. https://doi.org/10.1002/cd.23219842305

Roth, W. M., & McGinn, M. (1998). Inscriptions: Toward a theory of representing as social practice. Review of Educational Research, 68 (1), 35–59.

Sakshaug, L. E., & Wohlhuter, K. A. (2010). Journey toward teaching mathematics through problem-solving. School Science and Mathematics, 110 (8), 397–409. https://doi.org/10.1111/j.1949-8594.2010.00051.x

Santoso, B., & Syarifuddin, H. (2020). Validity of mathematic learning teaching administration on realistic mathematics education based approach to improve problem-solving. Journal of Physics. Conference Series, 1554 (1), 12001. https://doi.org/10.1088/1742-6596/1554/1/012001

Schoenfeld, A. H. (1980). Teaching problem-solving skills. The American Mathematical Monthly, 87 (10), 794. https://doi.org/10.2307/2320787

Schoenfeld, A. H. (1983). Problem solving in the mathematics curriculum . The Mathematical Association of America.

Schoenfeld, A. H. (2013). Reflections on problem-solving theory and practice. The Mathematics Enthusiast, 10 (1/2), 9.

Schoenfeld, A. H. (2016). Learning to think mathematically: Problem-solving, metacognition, and sense making in mathematics (Reprint). Journal of Education, 196 (2), 1–38. https://doi.org/10.1177/002205741619600202

Schoenfeld, A. H., Floden, R. E., & The algebra teaching study and mathematics assessment project. (2014). An introduction to the TRU Math document suite . Berkeley, CA & E. Lansing, MI: Graduate School of Education, University of California, Berkeley & College of Education, Michigan State University. Retrieved from: http://ats.berkeley.edu/tools.html

Schreier, M. (2012). Qualitative content analysis in practice . London: SAGE.

Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104 , 333–339. https://doi.org/10.1016/j.jbusres.2019.07.039

Stone, C. A. (1998). Should we salvage the scaffolding metaphor? Journal of Learning Disabilities, 31 (4), 409–413. https://doi.org/10.1177/002221949803100411

Stylianou, D. A. (2010). Teachers’ conceptions of representation in middle school mathematics. Journal of Mathematics Teacher Education, 13 (4), 325–343. https://doi.org/10.1007/s10857-010-9143-y

Sweller, J., Van Merrienboer, J. J. G., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31 (2), 261–292. https://doi.org/10.1007/s10648-019-09465-5

Sztajn, P., Borko, H., & Smith, T. (2017). Research on mathematics professional development. In J. Cai (Ed.), Compendium for research in mathematics education (Chapter 29, pp. 213–243). Reston, VA: National Council of Teachers of Mathematics.

Tobin, G. A., & Begley, C. M. (2004). Methodological rigor within a qualitative framework. Journal of Advanced Nursing, 48 , 388–396. https://doi.org/10.1111/j.1365-2648.2004.03207.x

Torraco, R. J. (2005). Writing integrative literature reviews: Guidelines and examples. Human Resource Development Review, 4 (3), 356–367. https://doi.org/10.1177/1534484305278283

Trouche, L., Gueudet, G., & Pepin, B. (2018). Documentational approach to didactics. In S. Lerman (Ed.), Encyclopedia of mathematics education. Cham: Springer. https://doi.org/10.1007/978-3-319-77487-9_100011-1

Trouche, L., Rocha, K., Gueudet, G., & Pepin, B. (2020). Transition to digital resources as a critical process in teachers’ trajectories: The case of Anna’s documentation work. ZDM Mathematics Education, 52 (7), 1243–1257. https://doi.org/10.1007/s11858-020-01164-8

Vale, I., & Barbosa, A. (2018). Mathematical problems: The advantages of visual strategies. Journal of the European Teacher Education Network, 13 , 23–33.

Vale, I., Pimentel, T., & Barbosa, A. (2018). The power of seeing in problem solving and creativity: An issue under discussion. In S. Carreira, N. Amado, & K. Jones (Eds.), Broadening the scope of research on mathematical problem-solving: A focus on technology, creativity and affect (pp. 243–272). Switzerland: Springer.

van Garderen, D., Scheuermann, A., Sadler, K., Hopkins, S., & Hirt, S. M. (2021). Preparing pre-service teachers to use visual representations as strategy to solve mathematics problems: What did they learn? Teacher Education and Special Education, 44 (4), 319–339. https://doi.org/10.1177/0888406421996070

Voskoglou, M. (2021). Problem solving and mathematical modelling. American Journal of Educational Research, 9 (2), 85–90. https://doi.org/10.12691/education-9-2-6

Vygotsky, L. S. (1978). Mind in society . Cambridge, MA: Harvard University Press.

Zahner, D., & Corter, J. E. (2010). The process of probability problem solving: Use of external visual representations. Mathematical Thinking and Learning, 12 (2), 177–204. https://doi.org/10.1080/10986061003654240

Zhang, J. (1997). The nature of external representations in problem solving. Cognitive Science, 21 (2), 179–217. https://doi.org/10.1207/s15516709cog2102_3

Download references

Acknowledgements

We would like to acknowledge and thank the teachers and students involved in the research.

Open Access funding enabled and organized by CAUL and its Member Institutions The study was financially supported by James Cook University Higher degree by research grant.

Author information

Authors and affiliations.

College of Arts, Society and Education, James Cook University, Cairns, Australia

Musarurwa David Chinofunga, Philemon Chigeza & Subhashni Taylor

You can also search for this author in PubMed   Google Scholar

Contributions

David: developed original idea, completed literature review, data analysis and authored the first draft of the article (80%). Philemon and Subhashni contributed to the data analysis, coherence of ideas and editing of the article (10% each).

Corresponding author

Correspondence to Musarurwa David Chinofunga .

Ethics declarations

Ethics approval.

This research was approved by the Human Research Ethics Committee, James Cook University (Approval Number H8201).

Informed consent

Informed consent was sought from all participants in accordance with the above ethics approval conditions. Before seeking consent from participants, permission was first obtained from the Queensland Department of Education and school principals of participating schools. The information sheets and consent forms were sent to participating mathematics teachers. An information sheet and consent form for students to share their artifacts was send through their teacher and consent was obtained directly from them as young adults (17–18 years). Consent forms and the data collected has been deidentified to protect participants and stored in the university repository.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A research report which is part of a PhD study by the first author who is an experienced high school mathematics teacher in Queensland, Australia. The second and third authors are primary and secondary advisors respectively. Correspondence concerning this article should be addressed to David Chinofunga, College of Arts, Society and Education, Nguma-bada Campus, Smithfield, Building A4, Cairns, PO Box 6811 Cairns QLD 4870, Australia.

Appendix 1 An approach to problem solving and mathematical modelling

Appendix 2 Phases three and four thematic analysis themes

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Chinofunga, M.D., Chigeza, P. & Taylor, S. How can procedural flowcharts support the development of mathematics problem-solving skills?. Math Ed Res J (2024). https://doi.org/10.1007/s13394-024-00483-3

Download citation

Received : 09 February 2023

Revised : 28 November 2023

Accepted : 06 January 2024

Published : 22 February 2024

DOI : https://doi.org/10.1007/s13394-024-00483-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Problem solving
• Procedural flowcharts
• Problem-solving stages
• Solution planning
• Visual representation
• Find a journal
• Publish with us
• Track your research

Your browser is not supported

Sorry but it looks as if your browser is out of date. To get the best experience using our site we recommend that you upgrade or switch browsers.

Find a solution

• Skip to main content
• Skip to navigation

• Back to parent navigation item
• Collections
• Sustainability in chemistry
• Simple rules
• Teacher well-being hub
• Women in chemistry
• Global science
• Escape room activities
• Decolonising chemistry teaching
• Teaching science skills
• Post-lockdown teaching support
• Get the print issue
• RSC Education

• More from navigation items

Source: © Sung Yoon Jo/Shutterstock

Practice makes perfect, but mix it up

By Fraser Scott 2022-03-24T08:30:00+00:00

• No comments

Does question bank design affect student performance?

We know it is important to provide students with repeated opportunities to practise solving questions. In a new study, researchers examined how the presentation and format of practice questions influences students’ problem-solving performance. The study revealed that mixed problem sets are better than questions arranged by topic.

There are two types of question practice teachers can give their students. The first, blocked practice, involves solving multiple problems of the same type, or about the same concept, before moving on to another. Practice worksheets and end-of-chapter questions in textbooks are often blocked practice type questions. They hone students’ algorithmic problem-solving skills, but at the expense of their conceptual understanding of the topic. The second, interleaved practice, shuffles between different types of questions in one session. It is more difficult, because students must identify the type of question being asked, or the concept it relates to, in addition to answering. Shuffled questions are thought to help with long-term learning and are similar to the questioning format in students’ exams.

The researchers investigated the effects of these two kinds of practice. They recruited 79 university students from general chemistry classes. They gave one group assignments with mixed questions and a control group assignments with questions organised into topics or chapters. They compared the groups’ performances through one pre-test and post-test after each of three problem-solving sessions.

Teaching tips

• Source or compile more mixed practice question banks, and avoid solely using topic-specific question sets from textbooks.
• To increase the difficulty and benefits of mixed problem question sets, do not include details identifying the relevant topics or chapters.
• Blocked practice still has a place. Certain topics, particularly those to do with numerical problem-solving, may require extensive blocked practice before students can engage with the benefits of interleaved practice.

Probing the problems

Rather than looking at overall scores, the researchers used a more detailed analysis. They broke each problem into the sub-problems, or steps, required to answer the problem. They then categorised students’ answers to the steps as successful, neutral or unsuccessful. Subcategories provided more insight into the students’ work. For example, the neutral category contained the subcategories ‘not required’, ‘did not know to do’ and ‘did something else’.

The study revealed that students in the interleaved-practice group increased their problem-solving success more than those in the blocked-practice group. Significantly, the achievement gap between the experimental and control groups widened as the study progressed. Following interleaved practice, students’ neutral codes decreased by about 70%, unsuccessful codes decreased by about 40%, and the successful codes increased by about 52%. Even if students were not able to complete the entire problem, they still improved at individual steps.

Importantly, even though A-, B- and C-grade students showed different levels of improvement, they all benefited from interleaved practice. Perhaps unexpectedly, B- and C-grade students improved the most. This might relate to their poorer conceptual understanding of topics or assessment literacy beforehand, which interleaved practice helps to develop.

Fraser Scott

O. Gulacar et al, Chem. Educ. Res. Pract., 2022, DOI: 10.1039/D1RP00334H

O. Gulacar et al,  Chem. Educ. Res. Pract.,  2022,  23 , 422–435 ( DOI: 10.1039/D1RP00334H )

• Action and context-based research
• Developing teaching practice
• Education research
• Problem solving

Related articles

Understanding how students untangle intermolecular forces

2024-03-14T05:10:00Z By Fraser Scott

Discover how learners use electronegativity to predict the location of dipole−dipole interactions 

Why I use video to teach chemistry concepts

2024-02-27T08:17:00Z By Helen Rogerson

Helen Rogerson shares why and when videos are useful in the chemistry classroom

3 ways to boost knowledge transfer and retention

2024-02-20T05:00:00Z By David Read

Apply these evidence-informed cognitive processes to ensure your learners get ahead

No comments yet

Only registered users can comment on this article., more from education research.

Boost maths skills to improve chemistry learning

2024-01-18T08:00:00Z By Fraser Scott

Use these evidence-based tips to help your learners get ahead with chemical calculations

Support student sensemaking through directed dialogue

2023-12-19T09:27:00Z By David Read

 Discover how to encourage effective classroom conversation to boost student understanding

How to banish misconceptions with green chemistry

2023-11-23T05:00:00Z By Fraser Scott

Evidence-informed tips on how a greener approach can improve problem-solving abilities and address misconceptions

• Contributors
• Print issue
• Email alerts

Site powered by Webvision Cloud

• Open access
• Published: 17 February 2022

Effectiveness of problem-based learning methodology in undergraduate medical education: a scoping review

• Joan Carles Trullàs   ORCID: orcid.org/0000-0002-7380-3475 1 , 2 , 3 ,
• Carles Blay   ORCID: orcid.org/0000-0003-3962-5887 1 , 4 ,
• Elisabet Sarri   ORCID: orcid.org/0000-0002-2435-399X 3 &
• Ramon Pujol   ORCID: orcid.org/0000-0003-2527-385X 1  

BMC Medical Education volume  22 , Article number:  104 ( 2022 ) Cite this article

31k Accesses

78 Citations

10 Altmetric

Metrics details

Problem-based learning (PBL) is a pedagogical approach that shifts the role of the teacher to the student (student-centered) and is based on self-directed learning. Although PBL has been adopted in undergraduate and postgraduate medical education, the effectiveness of the method is still under discussion. The author’s purpose was to appraise available international evidence concerning to the effectiveness and usefulness of PBL methodology in undergraduate medical teaching programs.

The authors applied the Arksey and O’Malley framework to undertake a scoping review. The search was carried out in February 2021 in PubMed and Web of Science including all publications in English and Spanish with no limits on publication date, study design or country of origin.

The literature search identified one hundred and twenty-four publications eligible for this review. Despite the fact that this review included many studies, their design was heterogeneous and only a few provided a high scientific evidence methodology (randomized design and/or systematic reviews with meta-analysis). Furthermore, most were single-center experiences with small sample size and there were no large multi-center studies. PBL methodology obtained a high level of satisfaction, especially among students. It was more effective than other more traditional (or lecture-based methods) at improving social and communication skills, problem-solving and self-learning skills. Knowledge retention and academic performance weren’t worse (and in many studies were better) than with traditional methods. PBL was not universally widespread, probably because requires greater human resources and continuous training for its implementation.

PBL is an effective and satisfactory methodology for medical education. It is likely that through PBL medical students will not only acquire knowledge but also other competencies that are needed in medical professionalism.

Peer Review reports

There has always been enormous interest in identifying the best learning methods. In the mid-twentieth century, US educator Edgar Dale proposed which actions would lead to deeper learning than others and published the well-known (and at the same time controversial) “Cone of Experience or Cone of Dale”. At the apex of the cone are oral representations (verbal descriptions, written descriptions, etc.) and at the base is direct experience (based on a person carrying out the activity that they aim to learn), which represents the greatest depth of our learning. In other words, each level of the cone corresponds to various learning methods. At the base are the most effective, participative methods (what we do and what we say) and at the apex are the least effective, abstract methods (what we read and what we hear) [ 1 ]. In 1990, psychologist George Miller proposed a framework pyramid to assess clinical competence. At the lowest level of the pyramid is knowledge (knows), followed by the competence (knows how), execution (shows how) and finally the action (does) [ 2 ]. Both Miller’s pyramid and Dale’s cone propose a very efficient way of training and, at the same time, of evaluation. Miller suggested that the learning curve passes through various levels, from the acquisition of theoretical knowledge to knowing how to put this knowledge into practice and demonstrate it. Dale stated that to remember a high percentage of the acquired knowledge, a theatrical representation should be carried out or real experiences should be simulated. It is difficult to situate methodologies such as problem-based learning (PBL), case-based learning (CBL) and team-based learning (TBL) in the context of these learning frameworks.

In the last 50 years, various university education models have emerged and have attempted to reconcile teaching with learning, according to the principle that students should lead their own learning process. Perhaps one of the most successful models is PBL that came out of the English-speaking environment. There are many descriptions of PBL in the literature, but in practice there is great variability in what people understand by this methodology. The original conception of PBL as an educational strategy in medicine was initiated at McMaster University (Canada) in 1969, leaving aside the traditional methodology (which is often based on lectures) and introducing student-centered learning. The new formulation of medical education proposed by McMaster did not separate the basic sciences from the clinical sciences, and partially abandoned theoretical classes, which were taught after the presentation of the problem. In its original version, PBL is a methodology in which the starting point is a problem or a problematic situation. The situation enables students to develop a hypothesis and identify learning needs so that they can better understand the problem and meet the established learning objectives [ 3 , 4 ]. PBL is taught using small groups (usually around 8–10 students) with a tutor. The aim of the group sessions is to identify a problem or scenario, define the key concepts identified, brainstorm ideas and discuss key learning objectives, research these and share this information with each other at subsequent sessions. Tutors are used to guide students, so they stay on track with the learning objectives of the task. Contemporary medical education also employs other small group learning methods including CBL and TBL. Characteristics common to the pedagogy of both CBL and TBL include the use of an authentic clinical case, active small-group learning, activation of existing knowledge and application of newly acquired knowledge. In CBL students are encouraged to engage in peer learning and apply new knowledge to these authentic clinical problems under the guidance of a facilitator. CBL encourages a structured and critical approach to clinical problem-solving, and, in contrast to PBL, is designed to allow the facilitator to correct and redirect students [ 5 ]. On the other hand, TBL offers a student-centered, instructional approach for large classes of students who are divided into small teams of typically five to seven students to solve clinically relevant problems. The overall similarities between PBL and TBL relate to the use of professionally relevant problems and small group learning, while the main difference relates to one teacher facilitating interactions between multiple self-managed teams in TBL, whereas each small group in PBL is facilitated by one teacher. Further differences are related to mandatory pre-reading assignments in TBL, testing of prior knowledge in TBL and activating prior knowledge in PBL, teacher-initiated clarifying of concepts that students struggled with in TBL versus students-generated issues that need further study in PBL, inter-team discussions in TBL and structured feedback and problems with related questions in TBL [ 6 ].

In the present study we have focused on PBL methodology, and, as attractive as the method may seem, we should consider whether it is really useful and effective as a learning method. Although PBL has been adopted in undergraduate and postgraduate medical education, the effectiveness (in terms of academic performance and/or skill improvement) of the method is still under discussion. This is due partly to the methodological difficulty in comparing PBL with traditional curricula based on lectures. To our knowledge, there is no systematic scoping review in the literature that has analyzed these aspects.

The main motivation for carrying out this research and writing this article was scientific but also professional interest. We believe that reviewing the state of the art of this methodology once it was already underway in our young Faculty of Medicine, could allow us to know if we were on the right track and if we should implement changes in the training of future doctors.

The primary goal of this study was to appraise available international evidence concerning to the effectiveness and usefulness of PBL methodology in undergraduate medical teaching programs. As the intention was to synthesize the scattered evidence available, the option was to conduct a scoping review. A scoping study tends to address broader topics where many different study designs might be applicable. Scoping studies may be particularly relevant to disciplines, such as medical education, in which the paucity of randomized controlled trials makes it difficult for researchers to undertake systematic reviews [ 7 , 8 ]. Even though the scoping review methodology is not widely used in medical education, it is well established for synthesizing heterogeneous research evidence [ 9 ].

The specific aims were: 1) to determine the effectiveness of PBL in academic performance (learning and retention of knowledge) in medical education; 2) to determine the effectiveness of PBL in other skills (social and communication skills, problem solving or self-learning) in medical education; 3) to know the level of satisfaction perceived by the medical students (and/or tutors) when they are taught with the PBL methodology (or when they teach in case of tutors).

This review was guided by Arksey and O’Malley’s methodological framework for conducting scoping reviews. The five main stages of the framework are: (1) identifying the research question; (2) ascertaining relevant studies; (3) determining study selection; (4) charting the data; and (5) collating, summarizing and reporting the results [ 7 ]. We reported our process according to the PRISMA Extension for Scoping Reviews [ 10 ].

Stage 1: Identifying the research question

With the goals of the study established, the four members of the research team established the research questions. The primary research question was “What is the effectiveness of PBL methodology for learning in undergraduate medicine?” and the secondary question “What is the perception and satisfaction of medical students and tutors in relation to PBL methodology?”.

Stage 2: Identifying relevant studies

After the research questions and a search strategy were defined, the searches were conducted in PubMed and Web of Science using the MeSH terms “problem-based learning” and “Medicine” (the Boolean operator “AND” was applied to the search terms). No limits were set on language, publication date, study design or country of origin. The search was carried out on 14th February 2021. Citations were uploaded to the reference manager software Mendeley Desktop (version 1.19.8) for title and abstract screening, and data characterization.

Stage 3: Study selection

The searching strategy in our scoping study generated a total of 2399 references. The literature search and screening of title, abstract and full text for suitability was performed independently by one author (JCT) based on predetermined inclusion criteria. The inclusion criteria were: 1) PBL methodology was the major research topic; 2) participants were undergraduate medical students or tutors; 3) the main outcome was academic performance (learning and knowledge retention); 4) the secondary outcomes were one of the following: social and communication skills, problem solving or self-learning and/or student/tutor satisfaction; 5) all types of studies were included including descriptive papers, qualitative, quantitative and mixed studies methods, perspectives, opinion, commentary pieces and editorials. Exclusion criteria were studies including other types of participants such as postgraduate medical students, residents and other health non-medical specialties such as pharmacy, veterinary, dentistry or nursing. Studies published in languages other than Spanish and English were also excluded. Situations in which uncertainty arose, all authors (CB, ES, RP) discussed the publication together to reach a final consensus. The outcomes of the search results and screening are presented in Fig.  1 . One-hundred and twenty-four articles met the inclusion criteria and were included in the final analysis.

Study flow PRISMA diagram. Details the review process through the different stages of the review; includes the number of records identified, included and excluded

Stage 4: Charting the data

A data extraction table was developed by the research team. Data extracted from each of the 124 publications included general publication details (year, author, and country), sample size, study population, design/methodology, main and secondary outcomes and relevant results and/or conclusions. We compiled all data into a single spreadsheet in Microsoft Excel for coding and analysis. The characteristics and the study subject of the 124 articles included in this review are summarized in Tables 1 and 2 . The detailed results of the Microsoft Excel file is also available in Additional file 1 .

Stage 5: Collating, summarizing and reporting the results

As indicated in the search strategy (Fig.  1 ) this review resulted in the inclusion of 124 publications. Publication years of the final sample ranged from 1990 to 2020, the majority of the publications (51, 41%) were identified for the years 2010–2020 and the years in which there were more publications were 2001, 2009 and 2015. Countries from the six continents were represented in this review. Most of the publications were from Asia (especially China and Saudi Arabia) and North America followed by Europe, and few studies were from Africa, Oceania and South America. The country with more publications was the United States of America ( n  = 27). The most frequent designs of the selected studies were surveys or questionnaires ( n  = 45) and comparative studies ( n  = 48, only 16 were randomized) with traditional or lecture-based learning methodologies (in two studies the comparison was with simulation) and the most frequently measured outcomes were academic performance followed by student satisfaction (48 studies measured more than one outcome). The few studies with the highest level of scientific evidence (systematic review and meta-analysis and randomized studies) were conducted mostly in Asian countries (Tables  1 and 2 ). The study subject was specified in 81 publications finding a high variability but at the same time great representability of almost all disciplines of the medical studies.

The sample size was available in 99 publications and the median [range] of the participants was 132 [14–2061]. According to study population, there were more participants in the students’ focused studies (median 134 and range 16–2061) in comparison with the tutors’ studies (median 53 and range 14–494).

Finally, after reviewing in detail the measured outcomes (main and secondary) according to the study design (Table 2 and Additional file 1 ) we present a narrative overview and a synthesis of the main findings.

Main outcome: academic performance (learning and knowledge retention)

Seventy-one of the 124 publications had learning and/or knowledge retention as a measured outcome, most of them ( n  = 45) were comparative studies with traditional or lecture-based learning and 16 were randomized. These studies were varied in their methodology, were performed in different geographic zones, and normally analyzed the experience of just one education center. Most studies ( n  = 49) reported superiority of PBL in learning and knowledge acquisition [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 ] but there was no difference between traditional and PBL curriculums in another 19 studies [ 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 ]. Only three studies reported that PBL was less effective [ 79 , 80 , 81 ], two of them were randomized (in one case favoring simulation-based learning [ 80 ] and another favoring lectures [ 81 ]) and the remaining study was based on tutors’ opinion rather than real academic performance [ 79 ]. It is noteworthy that the four systematic reviews and meta-analysis included in this scoping review, all carried out in China, found that PBL was more effective than lecture-based learning in improving knowledge and other skills (clinical, problem-solving, self-learning and collaborative) [ 40 , 51 , 53 , 58 ]. Another relevant example of the superiority of the PBL method over the traditional method is the experience reported by Hoffman et al. from the University of Missouri-Columbia. The authors analyzed the impact of implementing the PBL methodology in its Faculty of Medicine and revealed an improvement in the academic results that lasted for over a decade [ 31 ].

Secondary outcomes

Social and communication skills.

We found five studies in this scoping review that focused on these outcomes and all of them described that a curriculum centered on PBL seems to instill more confidence in social and communication skills among students. Students perceived PBL positively for teamwork, communication skills and interpersonal relations [ 44 , 45 , 67 , 75 , 82 ].

Student satisfaction

Sixty publications analyzed student satisfaction with PBL methodology. The most frequent methodology were surveys or questionnaires (30 studies) followed by comparative studies with traditional or lecture-based methodology (19 studies, 7 of them were randomized). Almost all the studies (51) have shown that PBL is generally well-received [ 11 , 13 , 18 , 19 , 20 , 21 , 22 , 26 , 29 , 34 , 37 , 39 , 41 , 42 , 46 , 50 , 56 , 58 , 63 , 64 , 66 , 78 , 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , 100 , 101 , 102 , 103 , 104 , 105 , 106 , 107 , 108 , 109 , 110 ] but in 9 studies the overall satisfaction scores for the PBL program were neutral [ 76 , 111 , 112 , 113 , 114 , 115 , 116 ] or negative [ 117 , 118 ]. Some factors that have been identified as key components for PBL to be successful include: a small group size, the use of scenarios of realistic cases and good management of group dynamics. Despite a mostly positive assessment of the PBL methodology by the students, there were some negative aspects that could be criticized or improved. These include unclear communication of the learning methodology, objectives and assessment method; bad management and organization of the sessions; tutors having little experience of the method; and a lack of standardization in the implementation of the method by the tutors.

Tutor satisfaction

There are only 15 publications that analyze the satisfaction of tutors, most of them surveys or questionnaires [ 85 , 88 , 92 , 98 , 108 , 110 , 119 ]. In comparison with the satisfaction of the students, here the results are more neutral [ 112 , 113 , 115 , 120 , 121 ] and even unfavorable to the PBL methodology in two publications [ 117 , 122 ]. PBL teaching was favored by tutors when the institutions train them in the subject, when there was administrative support and adequate infrastructure and coordination [ 123 ]. In some experiences, the PBL modules created an unacceptable toll of anxiety, unhappiness and strained relations.

Other skills (problem solving and self-learning)

The effectiveness of the PBL methodology has also been explored in other outcomes such as the ability to solve problems and to self-directed learning. All studies have shown that PBL is more effective than lecture-based learning in problem-solving and self-learning skills [ 18 , 24 , 40 , 48 , 67 , 75 , 93 , 104 , 124 ]. One single study found a poor accuracy of the students’ self-assessment when compared to their own performance [ 125 ]. In addition, there are studies that support PBL methodology for integration between basic and clinical sciences [ 126 ].

Finally, other publications have reported the experience of some faculties in the implementation of the PBL methodology. Different experiences have demonstrated that it is both possible and feasible to shift from a traditional curriculum to a PBL program, recognizing that PBL methodology is complex to plan and structure, needs a large number of human and material resources, requiring an immense teacher effort [ 28 , 31 , 94 , 127 , 128 , 129 , 130 , 131 , 132 , 133 ]. In addition, and despite its cost implication, a PBL curriculum can be successfully implemented in resource-constrained settings [ 134 , 135 ].

We conducted this scoping review to explore the effectiveness and satisfaction of PBL methodology for teaching in undergraduate medicine and, to our knowledge, it is the only study of its kind (systematic scoping review) that has been carried out in the last years. Similarly, Vernon et al. conducted a meta-analysis of articles published between 1970 and 1992 and their results generally supported the superiority of the PBL approach over more traditional methods of medical education [ 136 ]. PBL methodology is implemented in medical studies on the six continents but there is more experience (or at least more publications) from Asian countries and North America. Despite its apparent difficulties on implementation, a PBL curriculum can be successfully implemented in resource-constrained settings [ 134 , 135 ]. Although it is true that the few studies with the highest level of scientific evidence (randomized studies and meta-analysis) were carried out mainly in Asian countries (and some in North America and Europe), there were no significant differences in the main results according to geographical origin.

In this scoping review we have included a large number of publications that, despite their heterogeneity, tend to show favorable results for the usefulness of the PBL methodology in teaching and learning medicine. The results tend to be especially favorable to PBL methodology when it is compared with traditional or lecture-based teaching methods, but when compared with simulation it is not so clear. There are two studies that show neutral [ 71 ] or superior [ 80 ] results to simulation for the acquisition of specific clinical skills. It seems important to highlight that the four meta-analysis included in this review, which included a high number of participants, show results that are clearly favorable to the PBL methodology in terms of knowledge, clinical skills, problem-solving, self-learning and satisfaction [ 40 , 51 , 53 , 58 ].

Regarding the level of satisfaction described in the surveys or questionnaires, the overall satisfaction rate was higher in the PBL students when compared with traditional learning students. Students work in small groups, allowing and promoting teamwork and facilitating social and communication skills. As sessions are more attractive and dynamic than traditional classes, this could lead to a greater degree of motivation for learning.

These satisfaction results are not so favorable when tutors are asked and this may be due to different reasons; first, some studies are from the 90s, when the methodology was not yet fully implemented; second, the number of tutors included in these studies is low; and third, and perhaps most importantly, the complaints are not usually due to the methodology itself, but rather due to lack of administrative support, and/or work overload. PBL methodology implies more human and material resources. The lack of experience in guided self-learning by lecturers requires more training. Some teachers may not feel comfortable with the method and therefore do not apply it correctly.

Despite how effective and/or attractive the PBL methodology may seem, some (not many) authors are clearly detractors and have published opinion articles with fierce criticism to this methodology. Some of the arguments against are as follows: clinical problem solving is the wrong task for preclinical medical students, self-directed learning interpreted as self-teaching is not appropriate in undergraduate medical education, relegation to the role of facilitators is a misuse of the faculty, small-group experience is inherently variable and sometimes dysfunctional, etc. [ 137 ].

In light of the results found in our study, we believe that PBL is an adequate methodology for the training of future doctors and reinforces the idea that the PBL should have an important weight in the curriculum of our medical school. It is likely that training through PBL, the doctors of the future will not only have great knowledge but may also acquire greater capacity for communication, problem solving and self-learning, all of which are characteristics that are required in medical professionalism. For this purpose, Koh et al. analyzed the effect that PBL during medical school had on physician competencies after graduation, finding a positive effect mainly in social and cognitive dimensions [ 138 ].

Despite its defects and limitations, we must not abandon this methodology and, in any case, perhaps PBL should evolve, adapt, and improve to enhance its strengths and improve its weaknesses. It is likely that the new generations, trained in schools using new technologies and methodologies far from lectures, will feel more comfortable (either as students or as tutors) with methodologies more like PBL (small groups and work focused on problems or projects). It would be interesting to examine the implementation of technologies and even social media into PBL sessions, an issue that has been poorly explorer [ 139 ].

Limitations

Scoping reviews are not without limitations. Our review includes 124 articles from the 2399 initially identified and despite our efforts to be as comprehensive as possible, we may have missed some (probably few) articles. Even though this review includes many studies, their design is very heterogeneous, only a few include a large sample size and high scientific evidence methodology. Furthermore, most are single-center experiences and there are no large multi-center studies. Finally, the frequency of the PBL sessions (from once or twice a year to the whole curriculum) was not considered, in part, because most of the revised studies did not specify this information. This factor could affect the efficiency of PBL and the perceptions of students and tutors about PBL. However, the adoption of a scoping review methodology was effective in terms of summarizing the research findings, identifying limitations in studies’ methodologies and findings and provided a more rigorous vision of the international state of the art.

Conclusions

This systematic scoping review provides a broad overview of the efficacy of PBL methodology in undergraduate medicine teaching from different countries and institutions. PBL is not a new teaching method given that it has already been 50 years since it was implemented in medicine courses. It is a method that shifts the leading role from teachers to students and is based on guided self-learning. If it is applied properly, the degree of satisfaction is high, especially for students. PBL is more effective than traditional methods (based mainly on lectures) at improving social and communication skills, problem-solving and self-learning skills, and has no worse results (and in many studies better results) in relation to academic performance. Despite that, its use is not universally widespread, probably because it requires greater human resources and continuous training for its implementation. In any case, more comparative and randomized studies and/or other systematic reviews and meta-analysis are required to determine which educational strategies could be most suitable for the training of future doctors.

Abbreviations

• Problem-based learning

Case-based learning

Team-based learning

References:

Dale E. Methods for analyzing the content of motion pictures. J Educ Sociol. 1932;6:244–50.

Google Scholar  

Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9 Suppl):S63–7. https://doi.org/10.1097/00001888-199009000-00045 .

Article   Google Scholar  

Bodagh N, Bloomfield J, Birch P, Ricketts W. Problem-based learning: a review. Br J Hosp Med (Lond). 2017;78:C167–70. https://doi.org/10.12968/hmed.2017.78.11.C167 .

- Branda LA. El abc del ABP: Lo esencial del aprendizaje basado en problemas. In: Fundación Dr. Esteve, Cuadernos de la fundación Dr. Antonio Esteve nº27: El aprendizaje basado en problemas en sus textos, pp.1–16. 2013. Barcelona.

Burgess A, Matar E, Roberts C, et al. Scaffolding medical student knowledge and skills: team-based learning (TBL) and case-based learning (CBL). BMC Med Educ. 2021;21:238. https://doi.org/10.1186/s12909-021-02638-3 .

Dolmans D, Michaelsen L, van Merriënboer J, van der Vleuten C. Should we choose between problem-based learning and team-based learning? No, combine the best of both worlds! Med Teach. 2015;37:354–9. https://doi.org/10.3109/0142159X.2014.948828 .

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. In J Soc Res Methodol. 2005;8:19–32. https://doi.org/10.1080/1364557032000119616 .

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69. https://doi.org/10.1186/1748-5908-5-69 .

Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5:371–85. https://doi.org/10.1002/jrsm.1123 .

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467–73. https://doi.org/10.7326/M18-0850 .

Sokas RK, Diserens D, Johnston MA. Integrating occupational-health into the internal medicine clerkship using problem-based learning. Clin Res. 1990;38:A735.

Richards BF, Ober KP, Cariaga-Lo L, et al. Ratings of students’ performances in a third-year internal medicine clerkship: a comparison between problem-based and lecture-based curricula. Acad Med. 1996;71:187–9. https://doi.org/10.1097/00001888-199602000-00028 .

Gresham CL, Philp JR. Problem-based learning in clinical medicine. Teach Learn Med. 1996;8:111–5. https://doi.org/10.1080/10401339609539776 .

Hill J, Rolfe IE, Pearson SA, Heathcote A. Do junior doctors feel they are prepared for hospital practice? A study of graduates from traditional and non-traditional medical schools. Med Educ. 1998;32:19–24. https://doi.org/10.1046/j.1365-2923.1998.00152.x .

Blake RL, Parkison L. Faculty evaluation of the clinical performances of students in a problem-based learning curriculum. Teach Learn Med. 1998;10:69–73. https://doi.org/10.1207/S15328015TLM1002\_3 .

Hmelo CE. Problem-based learning: effects on the early acquisition of cognitive skill in medicine. J Learn Sc. 1998;7:173–208. https://doi.org/10.1207/s15327809jls0702\_2 .

Finch PN. The effect of problem-based learning on the academic performance of students studying podiatric medicine in Ontario. Med Educ. 1999;33:411–7.

Casassus P, Hivon R, Gagnayre R, d’Ivernois JF. An initial experiment in haematology instruction using the problem-based learning method in third-year medical training in France. Hematol Cell Ther. 1999;41:137–44. https://doi.org/10.1007/s00282-999-0137-0 .

Purdy RA, Benstead TJ, Holmes DB, Kaufman DM. Using problem-based learning in neurosciences education for medical students. Can J Neurol Sci. 1999;26:211–6. https://doi.org/10.1017/S0317167100000287 .

Farrell TA, Albanese MA, Pomrehn PRJ. Problem-based learning in ophthalmology: a pilot program for curricular renewal. Arch Ophthalmol. 1999;117:1223–6. https://doi.org/10.1001/archopht.117.9.1223 .

Curtis JA, Indyk D, Taylor B. Successful use of problem-based learning in a third-year pediatric clerkship. Ambul Pediatr. 2001;1:132–5. https://doi.org/10.1367/1539-4409(2001)001%3c0132:suopbl%3e2.0.co;2 .

Trevena LJ, Clarke RM. Self-directed learning in population health. a clinically relevant approach for medical students. Am J Prev Med. 2002;22:59–65. https://doi.org/10.1016/s0749-3797(01)00395-6 .

Astin J, Jenkins T, Moore L. Medical students’ perspective on the teaching of medical statistics in the undergraduate medical curriculum. Stat Med. 2002;21:1003–7. https://doi.org/10.1002/sim.1132 .

Whitfield CR, Manger EA, Zwicker J, Lehman EB. Differences between students in problem-based and lecture-based curricula measured by clerkship performance ratings at the beginning of the third year. Teach Learn Med. 2002;14:211–7. https://doi.org/10.1207/S15328015TLM1404\_2 .

McParland M, Noble LM, Livingston G. The effectiveness of problem-based learning compared to traditional teaching in undergraduate psychiatry. Med Educ. 2004;38:859–67. https://doi.org/10.1111/j.1365-2929.2004.01818.x .

Casey PM, Magrane D, Lesnick TG. Improved performance and student satisfaction after implementation of a problem-based preclinical obstetrics and gynecology curriculum. Am J Obstet Gynecol. 2005;193:1874–8. https://doi.org/10.1016/j.ajog.2005.07.061 .

Gurpinar E, Musal B, Aksakoglu G, Ucku R. Comparison of knowledge scores of medical students in problem-based learning and traditional curriculum on public health topics. BMC Med Educ. 2005;5:7. https://doi.org/10.1186/1472-6920-5-7 .

Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Effect of a community oriented problem based learning curriculum on quality of primary care delivered by graduates: historical cohort comparison study. BMJ. 2005;331:1002. https://doi.org/10.1136/bmj.38636.582546.7C .

Abu-Hijleh MF, Chakravarty M, Al-Shboul Q, Kassab S, Hamdy H. Integrating applied anatomy in surgical clerkship in a problem-based learning curriculum. Surg Radiol Anat. 2005;27:152–7. https://doi.org/10.1007/s00276-004-0293-4 .

Distlehorst LH, Dawson E, Robbs RS, Barrows HS. Problem-based learning outcomes: the glass half-full. Acad Med. 2005;80:294–9. https://doi.org/10.1097/00001888-200503000-00020 .

Hoffman K, Hosokawa M, Blake R Jr, Headrick L, Johnson G. Problem-based learning outcomes: ten years of experience at the University of Missouri-Columbia school of medicine. Acad Med. 2006;81:617–25. https://doi.org/10.1097/01.ACM.0000232411.97399.c6 .

Kong J, Li X, Wang Y, Sun W, Zhang J. Effect of digital problem-based learning cases on student learning outcomes in ophthalmology courses. Arch Ophthalmol. 2009;127:1211–4. https://doi.org/10.1001/archophthalmol.2009.110 .

Tsou KI, Cho SL, Lin CS, et al. Short-term outcomes of a near-full PBL curriculum in a new Taiwan medical school. Kaohsiung J Med Sci. 2009;25:282–93. https://doi.org/10.1016/S1607-551X(09)70075-0 .

Wang J, Zhang W, Qin L, et al. Problem-based learning in regional anatomy education at Peking University. Anat Sci Educ. 2010;3:121–6. https://doi.org/10.1002/ase.151 .

Abou-Elhamd KA, Rashad UM, Al-Sultan AI. Applying problem-based learning to otolaryngology teaching. J Laryngol Otol. 2011;125:117–20. https://doi.org/10.1017/S0022215110001702 .

Urrutia Aguilar ME, Hamui-Sutton A, Castaneda Figueiras S, van der Goes TI, Guevara-Guzman R. Impact of problem-based learning on the cognitive processes of medical students. Gac Med Mex. 2011;147:385–93.

Tian J-H, Yang K-H, Liu A-P. Problem-based learning in evidence-based medicine courses at Lanzhou University. Med Teach. 2012;34:341. https://doi.org/10.3109/0142159X.2011.531169 .

Hoover CR, Wong CC, Azzam A. From primary care to public health: using problem-based Learning and the ecological model to teach public health to first year medical students. J Community Health. 2012;37:647–52. https://doi.org/10.1007/s10900-011-9495-y .

Li J, Li QL, Li J, et al. Comparison of three problem-based learning conditions (real patients, digital and paper) with lecture-based learning in a dermatology course: a prospective randomized study from China. Med Teach. 2013;35:e963–70. https://doi.org/10.3109/0142159X.2012.719651 .

Ding X, Zhao L, Chu H, et al. Assessing the effectiveness of problem-based learning of preventive medicine education in China. Sci Rep. 2014;4:5126. https://doi.org/10.1038/srep05126 .

Meo SA. Undergraduate medical student’s perceptions on traditional and problem based curricula: pilot study. J Pak Med Assoc. 2014;64:775–9.

Khoshnevisasl P, Sadeghzadeh M, Mazloomzadeh S, Hashemi Feshareki R, Ahmadiafshar A. Comparison of problem-based learning with lecture-based learning. Iran Red Crescent Med J. 2014;16: e5186. https://doi.org/10.5812/ircmj.5186 .

Al-Drees AA, Khalil MS, Irshad M, Abdulghani HM. Students’ perception towards the problem based learning tutorial session in a system-based hybrid curriculum. Saudi Med J. 2015;36:341–8. https://doi.org/10.15537/smj.2015.3.10216 .

Al-Shaikh G, Al Mussaed EM, Altamimi TN, Elmorshedy H, Syed S, Habib F. Perception of medical students regarding problem based learning. Kuwait Med J. 2015;47:133–8.

Hande S, Mohammed CA, Komattil R. Acquisition of knowledge, generic skills and attitudes through problem-based learning: student perspectives in a hybrid curriculum. J Taibah Univ Medical Sci. 2015;10:21–5. https://doi.org/10.1016/j.jtumed.2014.01.008 .

González Mirasol E, Gómez García MT, Lobo Abascal P, Moreno Selva R, Fuentes Rozalén AM, González MG. Analysis of perception of training in graduates of the faculty of medicine at Universidad de Castilla-Mancha. Eval Program Plann. 2015;52:169–75. https://doi.org/10.1016/j.evalprogplan.2015.06.001 .

Yanamadala M, Kaprielian VS, O’Connor Grochowski C, Reed T, Heflin MT. A problem-based learning curriculum in geriatrics for medical students. Gerontol Geriatr Educ. 2018;39:122–31. https://doi.org/10.1080/02701960.2016.1152268 .

Balendran K, John L. Comparison of learning outcomes in problem based learning and lecture based learning in teaching forensic medicine. J Evol Med Dent Sci. 2017;6:89–92. https://doi.org/10.14260/jemds/2017/22 .

Chang H-C, Wang N-Y, Ko W-R, Yu Y-T, Lin L-Y, Tsai H-F. The effectiveness of clinical problem-based learning model of medico-jurisprudence education on general law knowledge for obstetrics/gynecological interns. Taiwan J Obstet Gynecol. 2017;56:325–30. https://doi.org/10.1016/j.tjog.2017.04.011 .

Eltony SA, El-Sayed NH, El-Araby SE-S, Kassab SE. Implementation and evaluation of a patient safety course in a problem-based learning program. Educ Heal. 2017;30:44–9. https://doi.org/10.4103/1357-6283.210512 .

Zhang S, Xu J, Wang H, Zhang D, Zhang Q, Zou L. Effects of problem-based learning in Chinese radiology education: a systematic review and meta-analysis. Medicine (Baltimore). 2018;97: e0069. https://doi.org/10.1097/MD.0000000000010069 .

Hincapie Parra DA, Ramos Monobe A, Chrino-Barcelo V. Problem based learning as an active learning strategy and its impact on academic performance and critical thinking of medical students. Rev Complut Educ. 2018;29:665–81. https://doi.org/10.5209/RCED.53581 .

Ma Y, Lu X. The effectiveness of problem-based learning in pediatric medical education in China: a meta-analysis of randomized controlled trials. Medicine (Baltimore). 2019;98: e14052. https://doi.org/10.1097/MD.0000000000014052 .

Berger C, Brinkrolf P, Ertmer C, et al. Combination of problem-based learning with high-fidelity simulation in CPR training improves short and long-term CPR skills: a randomised single blinded trial. BMC Med Educ. 2019;19:180. https://doi.org/10.1186/s12909-019-1626-7 .

Aboonq M, Alquliti A, Abdulmonem I, Alpuq N, Jalali K, Arabi S. Students’ approaches to learning and perception of learning environment: a comparison between traditional and problem-based learning medical curricula. Indo Am J Pharm Sci. 2019;6:3610–9. https://doi.org/10.5281/zenodo.2562660 .

Li X, Xie F, Li X, et al. Development, application, and evaluation of a problem-based learning method in clinical laboratory education. Clin Chim ACTA. 2020;510:681–4. https://doi.org/10.1016/j.cca.2020.08.037 .

Zhao W, He L, Deng W, Zhu J, Su A, Zhang Y. The effectiveness of the combined problem-based learning (PBL) and case-based learning (CBL) teaching method in the clinical practical teaching of thyroid disease. BMC Med Educ. 2020;20:381. https://doi.org/10.1186/s12909-020-02306 .

Liu C-X, Ouyang W-W, Wang X-W, Chen D, Jiang Z-L. Comparing hybrid problem-based and lecture learning (PBL plus LBL) with LBL pedagogy on clinical curriculum learning for medical students in China: a meta-analysis of randomized controlled trials. Medicine (Baltimore). 2020;99:e19687. https://doi.org/10.1097/MD.0000000000019687 .

Margolius SW, Papp KK, Altose MD, Wilson-Delfosse AL. Students perceive skills learned in pre-clerkship PBL valuable in core clinical rotations. Med Teach. 2020;42:902–8. https://doi.org/10.1080/0142159X.2020.1762031 .

Schwartz RW, Donnelly MB, Nash PP, Young B. Developing students cognitive skills in a problem-based surgery clerkship. Acad Med. 1992;67:694–6. https://doi.org/10.1097/00001888-199210000-00016 .

Mennin SP, Friedman M, Skipper B, Kalishman S, Snyder J. Performances on the NBME-I, NBME-II, and NBME-III by medical-students in the problem-based learning and conventional tracks at the university-of-new-mexico. Acad Med. 1993;68:616–24. https://doi.org/10.1097/00001888-199308000-00012 .

Kaufman DM, Mann KV. Comparing achievement on the medical council of Canada qualifying examination part I of students in conventional and problem-based learning curricula. Acad Med. 1998;73:1211–3. https://doi.org/10.1097/00001888-199811000-00022 .

Kaufman DM, Mann KV. Achievement of students in a conventional and Problem-Based Learning (PBL) curriculum. Adv Heal Sci Educ. 1999;4:245–60. https://doi.org/10.1023/A:1009829831978 .

Antepohl W, Herzig S. Problem-based learning versus lecture-based learning in a course of basic pharmacology: a controlled, randomized study. Med Educ. 1999;33:106–13. https://doi.org/10.1046/j.1365-2923.1999.00289.x .

Dyke P, Jamrozik K, Plant AJ. A randomized trial of a problem-based learning approach for teaching epidemiology. Acad Med. 2001;76:373–9. https://doi.org/10.1097/00001888-200104000-00016 .

Brewer DW. Endocrine PBL in the year 2000. Adv Physiol Educ. 2001;25:249–55. https://doi.org/10.1152/advances.2001.25.4.249 .

Seneviratne RD, Samarasekera DD, Karunathilake IM, Ponnamperuma GG. Students’ perception of problem-based learning in the medical curriculum of the faculty of medicine, University of Colombo. Ann Acad Med Singapore. 2001;30:379–81.

Alleyne T, Shirley A, Bennett C, et al. Problem-based compared with traditional methods at the faculty of medical sciences, University of the West Indies: a model study. Med Teach. 2002;24:273–9. https://doi.org/10.1080/01421590220125286 .

Norman GR, Wenghofer E, Klass D. Predicting doctor performance outcomes of curriculum interventions: problem-based learning and continuing competence. Med Educ. 2008;42:794–9. https://doi.org/10.1111/j.1365-2923.2008.03131.x .

Cohen-Schotanus J, Muijtjens AMM, Schoenrock-Adema J, Geertsma J, van der Vleuten CPM. Effects of conventional and problem-based learning on clinical and general competencies and career development. Med Educ. 2008;42:256–65. https://doi.org/10.1111/j.1365-2923.2007.02959.x .

Wenk M, Waurick R, Schotes D, et al. Simulation-based medical education is no better than problem-based discussions and induces misjudgment in self-assessment. Adv Health Sci Educ Theory Pract. 2009;14:159–71. https://doi.org/10.1007/s10459-008-9098-2 .

Collard A, Gelaes S, Vanbelle S, et al. Reasoning versus knowledge retention and ascertainment throughout a problem-based learning curriculum. Med Educ. 2009;43:854–65. https://doi.org/10.1111/j.1365-2923.2009.03410.x .

Nouns Z, Schauber S, Witt C, Kingreen H, Schuettpelz-Brauns K. Development of knowledge in basic sciences: a comparison of two medical curricula. Med Educ. 2012;46:1206–14. https://doi.org/10.1111/medu.12047 .

Saloojee S, van Wyk J. The impact of a problem-based learning curriculum on the psychiatric knowledge and skills of final-year students at the Nelson R Mandela school of medicine. South African J Psychiatry. 2012;18:116.

Mughal AM, Shaikh SH. Assessment of collaborative problem solving skills in undergraduate medical students at Ziauddin college of medicine. Karachi Pakistan J Med Sci. 2018;34:185–9. https://doi.org/10.12669/pjms.341.13485 .

Hu X, Zhang H, Song Y, et al. Implementation of flipped classroom combined with problem-based learning: an approach to promote learning about hyperthyroidism in the endocrinology internship. BMC Med Educ. 2019;19:290. https://doi.org/10.1186/s12909-019-1714-8 .

Thompson KL, Gendreau JL, Strickling JE, Young HE. Cadaveric dissection in relation to problem-based learning case sequencing: a report of medical student musculoskeletal examination performances and self-confidence. Anat Sci Educ. 2019;12:619–26. https://doi.org/10.1002/ase.1891 .

Chang G, Cook D, Maguire T, Skakun E, Yakimets WW, Warnock GL. Problem-based learning: its role in undergraduate surgical education. Can J Surg. 1995;38:13–21.

Vernon DTA, Hosokawa MC. Faculty attitudes and opinions about problem-based learning. Acad Med. 1996;71:1233–8. https://doi.org/10.1097/00001888-199611000-00020 .

Steadman RH, Coates WC, Huang YM, et al. Simulation-based training is superior to problem-based learning for the acquisition of critical assessment and management skills. Crit Care Med. 2006;34:151–7. https://doi.org/10.1097/01.CCM.0000190619.42013.94 .

Johnston JM, Schooling CM, Leung GM. A randomised-controlled trial of two educational modes for undergraduate evidence-based medicine learning in Asia. BMC Med Educ. 2009;9:63. https://doi.org/10.1186/1472-6920-9-63 .

Suleman W, Iqbal R, Alsultan A, Baig SM. Perception of 4(th) year medical students about problem based learning. Pakistan J Med Sci. 2010;26:871–4.

Blosser A, Jones B. Problem-based learning in a surgery clerkship. Med Teach. 1991;13:289–93. https://doi.org/10.3109/01421599109089907 .

Usherwood T, Joesbury H, Hannay D. Student-directed problem-based learning in general-practice and public-health medicine. Med Educ. 1991;25:421–9. https://doi.org/10.1111/j.1365-2923.1991.tb00090.x .

Bernstein P, Tipping J, Bercovitz K, Skinner HA. Shifting students and faculty to a PBL curriculum - attitudes changed and lessons learned. Acad Med. 1995;70:245–7. https://doi.org/10.1097/00001888-199503000-00019 .

Kaufman DM, Mann KV. Comparing students’ attitudes in problem-based and conventional curricula. Acad Med. 1996;71:1096–9. https://doi.org/10.1097/00001888-199610000-00018 .

Kalaian HA, Mullan PB. Exploratory factor analysis of students’ ratings of a problem-based learning curriculum. Acad Med. 1996;71:390–2. https://doi.org/10.1097/00001888-199604000-00019 .

Vincelette J, Lalande R, Delorme P, Goudreau J, Lalonde V, Jean P. A pilot course as a model for implementing a PBL curriculum. Acad Med. 1997;72:698–701. https://doi.org/10.1097/00001888-199708000-00015 .

Ghosh S, Dawka V. Combination of didactic lecture with problem-based learning sessions in physiology teaching in a developing medical college in Nepal. Adv Physiol Educ. 2000;24:8–12.

Walters MR. Problem-based learning within endocrine physiology lectures. Adv Physiol Educ. 2001;25:225–7. https://doi.org/10.1152/advances.2001.25.4.225 .

Leung GM, Lam TH, Hedley AJ. Problem-based public health learning - from the classroom to the community. Med Educ. 2001;35:1071–2.

Khoo HE, Chhem RK, Gwee MCE, Balasubramaniam P. Introduction of problem-based learning in a traditional medical curriculum in Singapore - students’ and tutors’ perspectives. Ann Acad Med Singapore. 2001;30:371–4.

Villamor MCA. Problem-based learning (PBL) as an approach in the teaching of biochemistry of the endocrine system at the Angeles University College of Medicine. Ann Acad Med Singapore. 2001;30:382–6.

Chang C-H, Yang C-Y, See L-C, Lui P-W. High satisfaction with problem-based learning for anesthesia. Chang Gung Med J. 2004;27:654–62.

McLean M. A comparison of students who chose a traditional or a problem-based learning curriculum after failing year 2 in the traditional curriculum: a unique case study at the Nelson R. Mandela school of medicine. Teach Learn Med. 2004;16:301–3. https://doi.org/10.1207/s15328015tlm1603\_15 .

Lucas M, García Guasch R, Moret E, Llasera R, Melero A. Canet J [Problem-based learning in an undergraduate medical school course on anesthesiology, recovery care, and pain management]. Rev Esp Anestesiol Reanim. 2006;53:419–25.

Burgun A, Darmoni S, Le Duff F, Weber J. Problem-based learning in medical informatics for undergraduate medical students: an experiment in two medical schools. Int J Med Inform. 2006;75:396–402. https://doi.org/10.1016/j.ijmedinf.2005.07.014 .

Gurpinar E, Senol Y, Aktekin MR. Evaluation of problem based learning by tutors and students in a medical faculty of Turkey. Kuwait Med J. 2009;41:123–7.

Elzubeir MA. Teaching of the renal system in an integrated, problem-based curriculum. Saudi J Kidney Dis Transpl. 2012;23:93–8.

Sulaiman N, Hamdy H. Problem-based learning: where are we now? Guide supplement 36.3–practical application. Med Teach. 2013;35:160–2. https://doi.org/10.3109/0142159X.2012.737965 .

Albarrak AI, Mohammed R, Abalhassan MF, Almutairi NK. Academic satisfaction among traditional and problem based learning medical students a comparative study. Saudi Med J. 2013;34:1179–88.

Nosair E, Mirghani Z, Mostafa RM. Measuring students’ perceptions of educational environment in the PBL program of Sharjah Medical College. J Med Educ Curric Dev. 2015;2:71–9. https://doi.org/10.4137/JMECDECDECD.S29926 .

Tshitenge ST, Ndhlovu CE, Ogundipe R. Evaluation of problem-based learning curriculum implementation in a clerkship rotation of a newly established African medical training institution: lessons from the University of Botswana. Pan Afr Med J. 2017;27:13. https://doi.org/10.11604/pamj.2017.27.13.10623 .

Yadav RL, Piryani RM, Deo GP, Shah DK, Yadav LK, Islam MN. Attitude and perception of undergraduate medical students toward the problem-based learning in Chitwan Medical College. Nepal Adv Med Educ Pract. 2018;9:317–22. https://doi.org/10.2147/AMEP.S160814 .

Asad MR, Tadvi N, Amir KM, Afzal K, Irfan A, Hussain SA. Medical student’s feedback towards problem based learning and interactive lectures as a teaching and learning method in an outcome-based curriculum. Int J Med Res & Heal Sci. 2019;8:78–84. https://doi.org/10.33844/ijol.2019.60392 .

Mpalanyi M, Nalweyiso ID, Mubuuke AG. Perceptions of radiography students toward problem-based learning almost two decades after its introduction at Makerere University. Uganda J Med imaging Radiat Sci. 2020;51:639–44. https://doi.org/10.1016/j.jmir.2020.06.009 .

Korkmaz NS, Ozcelik S. Evaluation of the opinions of the first, second and third term medical students about problem based learning sessions in Bezmialem Vakif University. Bezmialem Sci. 2020;8:144–9. https://doi.org/10.14235/bas.galenos.2019.3471 .

McGrew MC, Skipper B, Palley T, Kaufman A. Student and faculty perceptions of problem-based learning on a family medicine clerkship. Fam Med. 1999;31:171–6.

Kelly AM. A problem-based learning resource in emergency medicine for medical students. J Accid Emerg Med. 2000;17:320–3. https://doi.org/10.1136/emj.17.5.320 .

Bui-Mansfield LT, Chew FS. Radiologists as clinical tutors in a problem-based medical school curriculum. Acad Radiol. 2001;8:657–63. https://doi.org/10.1016/S1076-6332(03)80693-1 .

Macallan DC, Kent A, Holmes SC, Farmer EA, McCrorie P. A model of clinical problem-based learning for clinical attachments in medicine. Med Educ. 2009;43:799–807. https://doi.org/10.1111/j.1365-2923.2009.03406.x .

Grisham JW, Martiniuk ALC, Negin J, Wright EP. Problem-based learning (PBL) and public health: an initial exploration of perceptions of PBL in Vietnam. Asia-Pacific J public Heal. 2015;27:NP2019-27. https://doi.org/10.1177/1010539512436875 .

Khan IA, Al-Swailmi FK. Perceptions of faculty and students regarding Problem Based Learning: a mixed methods study. J Pak Med Assoc. 2015;65:1334–8.

Alduraywish AA, Mohager MO, Alenezi MJ, Nail AM, Aljafari AS. Evaluation of students’ experience with Problem-based Learning (PBL) applied at the College of Medicine, Al-Jouf University. Saudi Arabia J Pak Med Assoc. 2017;67:1870–3.

Yoo DM, Cho AR, Kim S. Satisfaction with and suitability of the problem-based learning program at the Catholic University of Korea College of Medicine. J Educ Eval Health Prof. 2019;16:20. https://doi.org/10.3352/jeehp.2019.16.20 .

Aldayel AA, Alali AO, Altuwaim AA, et al. Problem-based learning: medical students’ perception toward their educational environment at Al-Imam Mohammad Ibn Saud Islamic University. Adv Med Educ Pract. 2019;10:95–104. https://doi.org/10.2147/AMEP.S189062 .

DeLowerntal E. An evaluation of a module in problem-based learning. Int J Educ Dev. 1996;16:303–7. https://doi.org/10.1016/0738-0593(96)00001-6 .

Tufts MA, Higgins-Opitz SB. What makes the learning of physiology in a PBL medical curriculum challenging? Student perceptions. Adv Physiol Educ. 2009;33:187–95. https://doi.org/10.1152/advan.90214.2008 .

Aboonq M. Perception of the faculty regarding problem-based learning as an educational approach in Northwestern Saudi Arabia. Saudi Med J. 2015;36:1329–35. https://doi.org/10.15537/smj.2015.11.12263 .

Subramaniam RM, Scally P, Gibson R. Problem-based learning and medical student radiology teaching. Australas Radiol. 2004;48:335–8. https://doi.org/10.1111/j.0004-8461.2004.01317.x .

Chang BJ. Problem-based learning in medical school: a student’s perspective. Ann Med Surg. 2016;12:88–9. https://doi.org/10.1016/j.amsu.2016.11.011 .

Griffith CD, Blue AV, Mainous AG, DeSimone PA. Housestaff attitudes toward a problem-based clerkship. Med Teach. 1996;18:133–4. https://doi.org/10.3109/01421599609034147 .

Navarro HN, Zamora SJ. The opinion of teachers about tutorial problem based learning. Rev Med Chil. 2014;142:989–97. https://doi.org/10.4067/S0034-98872014000800006 .

Demiroren M, Turan S, Oztuna D. Medical students’ self-efficacy in problem-based learning and its relationship with self-regulated learning. Med Educ Online. 2016;21:30049. https://doi.org/10.3402/meo.v21.30049 .

Tousignant M, DesMarchais JE. Accuracy of student self-assessment ability compared to their own performance in a problem-based learning medical program: a correlation study. Adv Heal Sci Educ. 2002;7:19–27. https://doi.org/10.1023/A:1014516206120 .

Brynhildsen J, Dahle LO, Behrbohm Fallsberg M, Rundquist I, Hammar M. Attitudes among students and teachers on vertical integration between clinical medicine and basic science within a problem-based undergraduate medical curriculum. Med Teach. 2002;24:286–8. https://doi.org/10.1080/01421590220134105 .

Desmarchais JE. A student-centered, problem-based curriculum - 5 years experience. Can Med Assoc J. 1993;148:1567–72.

Doig K, Werner E. The marriage of a traditional lecture-based curriculum and problem-based learning: are the offspring vigorous? Med Teach. 2000;22:173–8.

Kemahli S. Hematology education in a problem-based curriculum. Hematology. 2005;10(Suppl 1):161–3. https://doi.org/10.1080/10245330512331390267 .

Grkovic I. Transition of the medical curriculum from classical to integrated: problem-based approach and Australian way of keeping academia in medicine. Croat Med J. 2005;46:16–20.

Bosch-Barrera J, Briceno Garcia HC, Capella D, et al. Teaching bioethics to students of medicine with Problem-Based Learning (PBL). Cuad Bioet. 2015;26:303–9.

Lin Y-C, Huang Y-S, Lai C-S, Yen J-H, Tsai W-C. Problem-based learning curriculum in medical education at Kaohsiung Medical University. Kaohsiung J Med Sci. 2009;25:264–9. https://doi.org/10.1016/S1607-551X(09)70072-5 .

Salinas Sánchez AS, Hernández Millán I, Virseda Rodríguez JA, et al. Problem-based learning in urology training the faculty of medicine of the Universidad de Castilla-La Mancha model. Actas Urol Esp. 2005;29:8–15. https://doi.org/10.1016/s0210-4806(05)73193-4 .

Amoako-Sakyi D, Amonoo-Kuofi H. Problem-based learning in resource-poor settings: lessons from a medical school in Ghana. BMC Med Educ. 2015;15:221. https://doi.org/10.1186/s12909-015-0501-4 .

Carrera LI, Tellez TE, D’Ottavio AE. Implementing a problem-based learning curriculum in an Argentinean medical school: implications for developing countries. Acad Med. 2003;78:798–801. https://doi.org/10.1097/00001888-200308000-00010 .

Vernon DT, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med. 1993;68:550–63. https://doi.org/10.1097/00001888-199307000-00015 .

Shanley PF. Viewpoint: leaving the “empty glass” of problem-based learning behind: new assumptions and a revised model for case study in preclinical medical education. Acad Med. 2007;82:479–85. https://doi.org/10.1097/ACM.0b013e31803eac4c .

Koh GC, Khoo HE, Wong ML, Koh D. The effects of problem-based learning during medical school on physician competency: a systematic review. CMAJ. 2008;178:34–41. https://doi.org/10.1503/cmaj.070565 .

Awan ZA, Awan AA, Alshawwa L, Tekian A, Park YS. Assisting the integration of social media in problem-based learning sessions in the faculty of medicine at King Abdulaziz University. Med Teach. 2018;40:S37–42. https://doi.org/10.1080/0142159X.2018.1465179 .

Download references

Acknowledgements

Not applicable

No funding was received for conducting this study.

Author information

Authors and affiliations.

Medical Education Cathedra, School of Medicine, University of Vic-Central University of Catalonia, Vic, Barcelona, Spain

Joan Carles Trullàs, Carles Blay & Ramon Pujol

Internal Medicine Service, Hospital de Olot i Comarcal de La Garrotxa, Olot, Girona, Spain

Joan Carles Trullàs

The Tissue Repair and Regeneration Laboratory (TR2Lab), University of Vic-Central University of Catalonia, Vic, Barcelona, Spain

Joan Carles Trullàs & Elisabet Sarri

Catalan Institute of Health (ICS) – Catalunya Central, Barcelona, Spain

Carles Blay

You can also search for this author in PubMed   Google Scholar

Contributions

JCT had the idea for the article, performed the literature search and data analysis and drafted the first version of the manuscript. CB, ES and RP contributed to the data analysis and suggested revisions to the manuscript. All authors read and approved the final manuscript.

Ethics declarations

Availability of data and materials.

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Ethics approval and consent to participate

Not applicable for a literature review.

Consent for publication

Competing interests.

All authors declare that they have no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Characteristics ofthe 124 included studies.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Trullàs, J.C., Blay, C., Sarri, E. et al. Effectiveness of problem-based learning methodology in undergraduate medical education: a scoping review. BMC Med Educ 22 , 104 (2022). https://doi.org/10.1186/s12909-022-03154-8

Download citation

Received : 03 October 2021

Accepted : 02 February 2022

Published : 17 February 2022

DOI : https://doi.org/10.1186/s12909-022-03154-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic review

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Donate Subscribe to our newsletter

Search form

Problem-solving cycle.

The Problem-Solving Cycle (PSC) is a National Science Foundation funded project that has developed a research-based professional development (PD) model.  This model is highly adaptable and can be specifically focused on problems of practice that are of interest to the participating teachers and administrators.  Additionally, it can be tailored to highlight federal, state, district, and school-based initiatives that are ever-changing and ongoing in the life of a teacher.

The PSC project is a research-practice partnership with the San Francisco Unified School District.  The current focus is on creating teacher leaders in middle school math classrooms and studying the effect on student learning.  

PSC Project Products:

Borko, H., Carlson, J., Jarry-Shore, M., Barnes, E., & Ellsworth, A. (2017, May). All students & teachers as math learners: A partnership to refine and implement two interconnected models. Presented at Stanford University’s CSET’s Pondering Excellence in Teaching Talk Series, Stanford, CA.

Borko, H., Carlson, J., Deutscher, R., & Ryan, J. (2018, May). A research-practice partnership to build district capacity. Video presented at 2018 STEM For All Video Showcase. http://stemforall2018.videohall.com/presentations/1299  

Borko, H. (2021 August). The Problem-Solving Cycle and Teacher Leadership Preparation Program: Developing and Researching a Model for Bringing Mathematics Professional Development to Scale . Research Seminar [Zoom] presented at IPN Leibniz Institute for Science and Mathematics Education, University of Kiel. 

Borko, H., Carlson, J., Mangram, C., Anderson, R., Fong, A., Million, S., Mozenter, S., & Villa, A. M. (2017). The role of video-based discussion in model for preparing professional development leaders. International Journal of STEM Education, 4 (1), 1-15.

Borko, H., Carlson, J., Deutscher, R., Boles, K. L., Delaney, V., Fong, A., Jarry-Shore, M., Malamut, J., Million, S., Mozenter, S., & Villa, A. M. (2021). Learning to Lead: an Approach to Mathematics Teacher Leader Development. International Journal of Science and Mathematics Education , 1-23.

Conference Presentations

Borko, H. (2015, February). Design-based implementation research in schools: Benefits & challenges . Paper presented at AACTE, Washington, D.C.

Borko, H., & Carlson, J. (2016, April) Design-based implementation research: adapting a professional development leadership model with a school district” Paper presented at AERA in a symposium entitled A Behind-the-Scenes Look at Effective Video-Based Professional Development , Washington, D.C.

Borko, H. (2016, June). Preparing mathematics teachers to facilitate the problem-solving cycle professional development . Paper presented at the Symposium and Workshop on Video Resources for Mathematics Teacher Development at the Weizmann Institute, Rehovot, Israel.

Mozenter, S. (2017, February). Video-based discussions: Meeting the multiple demands of PD for content teachers serving English language learners. Presented at National Association for Bilingual Education, Dallas, TX.

Borko, H., & Villa III, A. M. (2017, March). Facilitating Video-Based Mathematics Professional Development. Presented at Teacher Development Group Leadership Seminar, Portland, OR.

Villa III, A. M., & Jarry-Shore, M. (2017, March). Facilitating video-based mathematics professional development. Research symposium at National Council of Teachers of Mathematics Research Conference, San Antonio, TX.

Carlson, J., Jarry-Shore, M., Barnes, E., & Ellsworth, A. (2017, March). All students & teachers as math learners: A partnership to refine and implement two interconnected models.   Presented at Stanford-SFUSD Partnership Annual Meeting, Stanford, CA.

Jarry-Shore, M., Fong, A., Dyer, E., Gomez Zaccarelli, F., & Borko, H. (2018, February).  Video for equity: Designing video-based discussions of student authority.  Presentation at Association of Mathematics Teacher Education, Houston, TX.

Fong, A., Dyer, E., & Gomez Zaccarelli, F. (2018, February).  A shared vision for teacher improvement: Adapting professional development for local context by leveraging district-developed tools.  Presentation at Association of Mathematics Teacher Education, Houston, TX.

Mozenter, S., Gomez Zaccarelli, F., & Ellsworth, A. (2018, February ).  Video-based discussions in service of student agency, authority, and identity. Presentation at the Association of Teacher Education, Las Vegas, NV.

Mozenter, S., Ellsworth, A., & Gomez Zaccarelli, F. (2018, March). Video-based discussions in service of student agency, authority, & identity. Presentation at the American Association of Colleges for Teacher Education, Baltimore. MD.

Borko, H., & Villa III, A. M. (2018, March ). Building district capacity to address student access & equity: A research-practice partnership to develop teacher leaders. Presentation at the Teacher Development Group Leadership Seminar, Portland, OR.

Borko, H., Carlson, J., & Treviño, E. (2018, April).  A research-practice partnership to develop district capacity: Learning with & from each other.  Paper presented at the American Educational Research Association, New York, NY. 

Mozenter, S., Borko, H., & Jarry-Shore, M. (2018, June). Complicating the connection: Immigrant-background teachers . Paper presented at Teaching & Teacher Education Special Interest Group of the European Association for Research on Learning and Instruction, Kristiansaand, Norway.

Treviño, E. Brown, A., Villa III, A.M., & Borko, H. (2018, November).  Deconstructing student math content knowledge and groupwork through video-based discussion. Presentation at California Mathematics Council - Northern Section Conference Asilomar, Pacific Grove, CA.

Jarry-Shore, M. (2018, November ). The in-the-moment noticing of the novice mathematics teacher. Paper and presentation at the North American chapter of the International Group for the Psychology of Mathematics Education, Greenville, SC.

Villa III, A.M., & Boles, K. (2019, February).  Actualizing agency, authority, identity, and access to content in two contrasting cases of mathematical groupwork . Presented at Association of Mathematics Teacher Education, Orlando, FL.

Borko, H., & Villa III, A.M. (2019, February/March). Building teachers’ capacity to promote students’ access to rigorous and meaningful mathematics through video-based discussions. Presentation at the Teacher Development Group Leadership Seminar, Portland, OR.

Gomez Zaccarelli, F., Villa III, A.M., Mozenter, S., Boles, K., Deutscher, R., Borko, H., & Carlson, J. (2019, April).  How students are oriented toward a mathematical task and their peers: Access to content, agency, authority, and identity. Paper presented at the American Educational Research Association, Toronto, Canada. 

Mozenter, S., & Borko, H. (2019, April ). “ Not many people ask me this kind of question.” Three contrasting cases of immigrant-background teachers . Paper presented at the American Educational Research Association, Toronto, Canada.

Borko, H., Carlson, J., & Deutscher, R. (2019,  April ). Learning environments to support teacher leaders’ learning to lead video-based discussions. Poster presented in the structured poster session at the American Educational Research Association, Toronto, Canada .

Villa III, A.M., Boles, K.L., & Borko, H. (2019, November ).  Teacher leader learning through participation in and facilitation of professional development addressing problems of practice . Paper and presentation at the North American chapter of the International Group for the Psychology of Mathematics Education, St. Louis, MO.

Boles, K. L., Jarry-Shore, M., Muro Villa III, A., Malamut, J., & Borko, H. (2020, June). Building capacity via facilitator agency: Tensions in implementing an adaptive model of professional development. In M. Gresalfi, & I. S. Horn (Eds.),  The Interdisciplinarity of the Learning Sciences, 14th International Conference of the Learning Sciences (ICLS)  (pp. 2585-2588). Nashville, TN: International Society of the Learning Sciences. 

Jarry-Shore, M., & Allen, T. (2020, December). Noticing Struggle to Support Student Understanding [Conference Presentation]. California Mathematics Council - North Conference, Pacific Grove, CA, United States.

• Stanford Home
• Maps & Directions
• Search Stanford
• Emergency Info
• Terms of Use
• Non-Discrimination
• Accessibility

© Stanford University , Stanford , California 94305 .

• Open access
• Published: 07 April 2024

Efficacy of a virtual nursing simulation-based education to provide psychological support for patients affected by infectious disease disasters: a randomized controlled trial

• Eunjung Ko 1 &
• Yun-Jung Choi 1  

BMC Nursing volume  23 , Article number:  230 ( 2024 ) Cite this article

1 Altmetric

Metrics details

Virtual simulation-based education for healthcare professionals has emerged as a strategy for dealing with infectious disease disasters, particularly when training at clinical sites is restricted due to the risk of infection and a lack of personal protective equipment. This research evaluated a virtual simulation-based education program intended to increase nurses’ perceived competence in providing psychological support to patients affected by infectious disease disasters.

The efficacy of the program was evaluated via a randomized controlled trial. We recruited 104 nurses for participation in the study and allocated them randomly and evenly to an experimental group and a control group. The experimental group was given a web address through which they could access the program, whereas the control group was provided with a web address that directed them to text-based education materials. Data were then collected through an online survey of competence in addressing disaster mental health, after which the data were analyzed using the Statistical Package for the Social Sciences(version 23.0).

The analysis showed that the experimental group’s disaster mental health competence (F = 5.149, p  =.026), problem solving process (t = 3.024, p  =.003), self-leadership (t = 2.063, p  =.042), learning self-efficacy (t = 3.450, p  =.001), and transfer motivation (t = 2.095, p  =.039) significantly statistically differed from those of the control group.

Conclusions

A virtual nursing simulation-based education program for psychological support can overcome limitations of time and space. The program would also be an effective learning resource during infectious disease outbreaks.

Clinical trial registration

This Korean clinical trial was retrospectively registered (21/11/2023) in the Clinical Research Information Service ( https://cris.nih.go.kr ) with trial registration number KCT0008965.

Peer Review reports

The last two decades have confronted the world with a variety of infectious diseases, such as severe acute respiratory syndrome, which first occurred in Asia in 2003 before spreading worldwide, including Korea, in only a few months. Since then, infectious disease outbreaks began to be recognized as severe disasters. Other examples include the 2009 H1N1 influenza outbreak, which caused more than 10,000 deaths worldwide and 140 deaths in Korea; the proliferation of the Ebola virus, which resulted in a fatality rate of more than 90% in Africa in 2014; and the outbreak of Middle East respiratory syndrome in 2015, Zika virus disease in 2016, and coronavirus disease (COVID-19) in 2019 [ 1 ]. The COVID-19 pandemic, in particular, has caused infections among approximately 64 million people and the deaths of 1.5 million individuals as of December 2020 [ 2 ].

Direct victims of infectious disease disasters, infected patients, and quarantined individuals suffer from a fear of stigma or social blame and guilt, but even people who are unexposed to sources of infection experience psychological distress from anxiety and fear of disease or possible death [ 3 ]. They also blame infected people and harbor hatred toward them [ 3 ]. This assertion is supported by an examination of web search behaviors and infodemic attitudes toward COVID-19, which identified superficial and racist attitudes [ 4 ]. Additionally, in research using a health stigma and discrimination framework related to communicable diseases, the authors found that people exhibit negative stereotypes, biases, and discriminatory conduct toward infected groups owing to fears of contagion, concerns about potential harm, and perceptions that individuals violate central values [ 5 ]. Stigmatized individuals experience adverse effects on their health because of both the stress induced by stigma and the decreased use of available services [ 5 ].

Severe and prolonged anxiety, fear, blame, and aggression can lead to mental health problems, including depression, anxiety, panic attacks, somatic symptoms, post-traumatic stress disorder, psychosis, and even suicide and life-threatening behaviors [ 6 ]. Therefore, recovery from the psychological trauma caused by a disaster should be regarded as equally necessary as physical recovery, with emphasis placed on psychological support activities that prevent the deterioration of mental health [ 7 ].

Disasters pose a significant threat to mental health support systems, wherein the lack of healthcare professionals or psychologists trained to address these conditions exacerbates the psychological distress and psychopathological risk experienced by society [ 8 ]. When training at clinical sites is restricted due to infection risks and a lack of personal protective equipment (PPE), an emerging solution is virtual simulation [ 9 ].

A virtual simulation is a simulation modality developed on the basis of video or graphic recordings featuring virtual patients and delivered via either a static or mobile device. It replicates real-world clinical situations and affords learners an interactive experience [ 10 ]. Virtual simulation-based education provides an immersive clinical environment, as virtual patients respond to a learner’s assessments and interventions [ 11 , 12 ]. It enables two-way communication, and allows medical professionals to practice making clinical decisions [ 10 ]. Virtual patients are equipped with voice, intonation, and expressions that reinforce the educational narrative within the virtual environment, thereby enhancing the effectiveness of the learning experience [ 13 ]. One of the primary advantages of virtual simulation-based education is its provision of a safe and non-threatening environment in which learners can practice. It also offers flexible and reproducible learning experiences, thus catering to the diverse needs of learners [ 14 ].

Self-assessment is the most commonly used competence evaluation tool, as it is cost-effective and helps nurses improve their practice by identifying their strengths and weaknesses for development [ 15 ]. Self-assessed competence is also related to the quality of patient care because nurses promote continuous learning by determining educational needs through such evaluations [ 16 ]. The competence perceived by a nurse is inherently subjective given its self-reported nature and poses a challenge in establishing a direct correlation with the actual care of patients [ 17 , 18 ]. However, studies have indicated that increased levels of self-perceived competence are associated with a significant increase in core competencies related to patient care and frequent use of clinical skills [ 19 , 20 ]. Perceived competence likewise influences the job satisfaction and organizational citizenship behavior of nurses and is significantly related to absenteeism, one of the deterrents to the delivery of quality care [ 21 , 22 ].

Competence refers to the possession of qualifications and abilities to satisfy professional standards, as well as the capability to perform tasks and duties in a suitable and effective manner [ 23 ]. Competencies for disaster mental health are crucial for enhancing disaster response capabilities. These competencies encompass a range of skills, knowledge, and attitudes necessary for mental health professionals to effectively support individuals and communities affected by disasters [ 24 ]. Such competencies and how they are affected by simulation-based training have been explored in some studies, which reported a significant increase in competence after exposure to the aforementioned education [ 25 , 26 ].

The simulation education defined in mock training designs based on real situations provides opportunities to exercise problem-solving through various strategies. Problem-solving process is considered key competency through which learners are expected to enhance their relevant knowledge and clinical performance abilities [ 27 ]. In particular, problem-solving processes for identifying and assessing problems and finding solutions are psychological strategies that help people cope and recover after a disaster [ 28 ]. A scoping review on the effect of simulation-based education on the problem-solving process indicated that out of 32 studies reviewed, 21 demonstrated statistically significant improvement in people’s ability to resolve problems [ 29 ].

Simulation training can also address self-leadership, which is an essential self-learning quality that aids individuals in staying motivated and focused on their learning goals. It is also required as a basic qualification of professional nurses, who must be able to take initiative and make responsible decisions [ 30 , 31 ]. Previous studies have reported statistically significant improvements in self-leadership following simulation training [ 32 , 33 ].

Another aspect that benefits from simulation-driven education is learning self-efficacy, which plays a crucial role in predicting learners’ levels of engagement and academic success in online education. It reflects learners’ confidence in their ability to manage their own learning process. It is a significant predictor of both learners’ participation levels and their academic achievements in online education settings [ 34 , 35 ]. Several studies have demonstrated virtual simulation- or online education-induced significant improvements in learning self-efficacy [ 36 , 37 ]. Finally, virtual simulation-based education can also improve the motivation to transfer new knowledge and skills learned through education to clinical practice [ 38 ]. This motivation is considered an essential measure of effective learning for nurses working in the clinical field [ 38 ]. A previous study reported that psychiatric nursing simulation training combined with post-course debriefing significantly increases participants’ level of motivation to transfer [ 38 ].

On the basis of the discussion above, this study evaluated a virtual nursing simulation-based education program on disaster psychology designed to provide psychological support to patients affected by infectious disease disasters.

Study design

This study conducted a randomized controlled trial (RCT) to test the virtual nursing simulation-based education program of interest. The RCT protocol used was based on CONSORT guidelines.

Participants

We recruited nurses working at general hospitals in South Korea. With permission from the nurse managers of these hospitals, a participation notice was posted on the institutions’ internet bulletin boards for nurses for a week. The two-sided test criterion, with a significance level (α) of 0.05, a power (1-β) of 0.80, and a medium effect size of 0.6, dictates that the minimum number of participants per group be 90. The effect size was based on a virtual simulation intervention study conducted by Kim and Choi [ 36 ]. Taking the dropout rate into consideration, we recruited 104 nurses, who were assigned to an experimental group and a control group using the random sampling functionality of the Statistical Package for the Social Sciences (SPSS version 23.0). Out of the initial sample, 11 participants were excluded because they were on vacation, could not be contacted, or provided incomplete responses during data collection (Fig.  1 ).

Flowchart of the randomized controlled trial

The virtual nursing simulation-based education program

This study probed into the virtual nursing simulation-based education program developed by Ko [ 39 ]. The program is implemented using an e-learning development platform, Articulate Storyline, whose operating environment is compatible with all web browsers (Internet Explorer, Microsoft Edge, Firefox, Google Chrome, etc.). It is a mobile-friendly application that can run in devices with Android and iOS operating systems. When an individual uses their smartphone or personal computer to access the server via the web address corresponding to the education program, the content functions execute. Ko’s [ 39 ] program involves five stages of learning completed in 100 min: (1) preparatory learning (30 min), (2) pre-test (5 min), (3) pre-briefing (5 min), (4) simulation game (30 min), and (5) structured self-debriefing (30 min) (Fig.  2 ).

Preparatory learning comes with lecture materials on guidelines for providing psychological support to victims of infectious disease disasters, administering psychological first aid, donning and doffing PPE, and exercising mindfulness through videos and pictures. In the pretest stage, a learner answers five questions and can immediately check the correct responses, which come with detailed explanations. In the prebriefing stage, an overview of a nursing simulation scenario, patient information, learning objectives, and instructions on using the virtual simulation are provided. During the simulation game, a video of the simulation is presented. It starts with a 39-year-old female, a standardized patient who is age- and gender-matched to the scenario, confirmed to have contracted COVID-19 and transferred to a negative pressure isolation room. The patient presents with extreme anxiety and feeling of tightness in her chest. During the game, learners are expected to complete 12 quizzes. In the debriefing stage, a summary of the simulation quiz results and self-debriefing questions are provided, and the comments made by learners are saved in the Naver cloud platform.

The evaluated virtual nursing simulation-based education program (examples are our own work)

Measurements

Disaster mental health competence.

Disaster mental health competence was measured using the perceived competence scale for disaster mental health workforce (PCS-DMHW), which was developed by Yoon and Choi [ 40 ]. This tool consists of 24 questions related to knowledge (6 questions), attitudes (9 questions), and skills (9 questions). Each item is rated using a five-point Likert scale (0 = strongly disagree, 4 = strongly agree), and the responses are summed. The higher the score, the greater the perception of competence in a relevant area [ 40 ]. The Cronbach’s α values of the PCS-DMHW were 0.95 and 0.94 at the time of tool development and the present study, respectively.

Problem solving process

Problem solving process was determined using a tool modified and supplemented by Park and Woo [ 41 ] on the grounds of the problem solving process and behavior survey developed by Lee [ 42 ]. This tool is composed of 25 questions on five factors, namely, problem discovery, problem definition, problem solution design, problem solution execution, and problem solving review [ 41 ]. The reliability of the tool was 0.89 at the time of development [ 41 ], but the Cronbach’s α found in the current research was 0.94.

Self-leadership

Self-leadership was measured using a tool developed by Manz [ 43 ] and modified by Kim [ 44 ]. The tool consists of 18 questions distributed over six factors (three questions each): self-defense, rehearsal, goal setting, self-compensation, self-expense edition, and constructive thinking. The reliability of the tool at the time of development and the present research was (Cronbach’s α) 0.87 and 0.82, respectively.

Learning self-efficacy

To ascertain learning self-efficacy, we used the tool developed by Ayres [45] and translated by Park and Kweon [ 38 ]. This tool consists of 10 questions, and it had a reliability (Cronbach’s ⍺) of 0.94 and 0.93 at the time of development and the current study, respectively.

Motivation to transfer

We used Ayres’s [45] motivation to transfer scale, which was translated by Park and Kweon [ 38 ]. Its reliability at the time of development and the present research was (Cronbach’s ⍺) 0.80 and 0.93, respectively.

Data collection

The experimental and control groups were administered a pretest through an online survey. The web address through which the evaluated virtual simulation-based education program could be accessed was provided to the experimental group, whereas text-based education materials on psychological support for victims of infectious disease disasters were given to the control group. The groups were simultaneously sent the program’s instruction manual, and their inquiries were answered through chat. After the interventions, each participant was administered a posttest through another online survey.

Data analysis

The collected data were analyzed using SPSS version 23.0. The homogeneity test for general characteristics between the experimental and control groups was analyzed using a t-test, a chi-square test, and Fisher’s exact test. The normality of the dependent variables was analyzed using the Kolmogorov-Smirnov test. Changes in the dependent variables between the pretest and posttest were analyzed using a paired t-test. Differences in the dependent variables before and after the groups’ use of the interventions were examined via a t-test and ANCOVA.

Ethical considerations

We completed education in bioethics law prior to the research and obtained approval of the research proposal and questionnaire from the Institutional Review Board of the affiliated university (IRB approval number 1041078-202003-HRSB-070-01CC). A signed consent form was also obtained from each participant after the purpose and methods of the research, the confidentiality of personal information, and the voluntary nature of participation or their right to withdraw from the study were explained to them. All collected data were kept in a lockable cabinet, and electronic data were encrypted and stored. These data are to be discarded after three years.

A total of 93 participants (45 in the experimental group and 48 in the control group) were left after the exclusion of unsuitable respondents. of the between-group comparisons of the subjects indicated no significant differences between them (5% significance level) in terms of general characteristics, such as gender, age, work unit, and clinical experience (Table  1 ).

The score of the experimental group on disaster mental health competence increased from 48.13 in the pretest to 70.51 in the posttest (+ 22.38), whereas that of the control group increased from 53.33 in the pretest to 68.38 in the posttest (+ 15.04). These findings reflect a statistically significant difference in competence between the groups (F = 5.149, p  =.026). The scores of the experimental and control groups on problem solving process increased from 73.07 in the pretest to 88.24 in the posttest (+ 15.18) and from 75.75 in the pretest to 83.77 in the posttest (+ 8.02), respectively. As with the competence findings, these point to a significant difference between the groups in terms of the ability to resolve problems (t = 3.024, p  =.003) (Table  2 ).

The score of the experimental group on self-leadership increased from 54.87 in the pretest to 59.58 in the posttest (+ 4.71), and that of the control group increased from 57.48 in the pretest to 60.10 in the posttest (+ 2.63). These results denote a statistically significant difference in this ability between the groups (t = 2.063, p  =.042). The scores of the experimental and control participants on learning self-rose from 55.40 in the pretest to 58.84 in the posttest (+ 3.44) and from 56.81 in the pretest to 57.13 in the posttest (+ 0.31), respectively. Again, a statistically significant difference was found between the groups (t = 3.450, p  =.001). Their scores on motivation to transfer rose from 49.31 in the pretest to 54.29 in the posttest (+ 4.98) (experimental group) and the score increased from 50.50 in the pretest to 51.85 in the posttest (+ 1.35) (control group), pointing to a significant difference between the groups (t = 2.095, p  =.039).

As previously stated, this research was evaluated a virtual nursing simulation-based education program designed to provide psychological support to patients affected by infectious disease disasters. The results showed statistically significant increases in the experimental group’s pretest and posttest scores on disaster mental health competence, problem solving process, self-leadership, learning self-efficacy, and motivation to transfer.

The experimental group achieved more statistically significant improvements in disaster mental health competence than did the control group. This finding is similar to the statistically significant increase in the average disaster mental health competence shown by providers of disaster mental health services providers and non-expert groups after PFA training involving lecture and practice [ 46 ]. It is also consistent with the significant increase in the scores of school counselors on disaster mental health competence after a lecture and simulation on PFA [ 25 ]. In their study on disaster relief workers, Kang and Choi [ 26 ] measured the participants’ performance competence in PFA after the delivery of a lecture and simulation-based education using a standardized patient. The authors found a significant increase in PFA performance competence, consistent with the present research. Since there are currently no other virtual simulation-based education programs for disaster psychological support available, we compared the effectiveness of various PFA training methods with the program assessed in the present work.

In the current research, the posttest scores of the experimental group on problem solving process significantly increased, similar to the results of Kim et al.’s study on virtual simulation- and blended simulation-based education on asthmatic child nursing [ 47 ]. Both the control and experimental groups (virtual simulation only and blended simulation featuring high-fidelity and virtual simulations, respectively) showed an increase in their problem solving process scores. These results and those derived in the present work are similar because reading and pretest phases were incorporated into the design of the previous study. Given that researchers have used commercial virtual simulations featuring avatars rather than standardized patient videos available through English-based platforms, user experiences may differ, thus requiring a qualitative analysis to identify differences. However, Kim et al. [ 47 ] did not implement a debriefing after the virtual simulation program, rendering comparison impossible. Another research reported that a multimodality simulation education that combines such methods as virtual simulation, the use of mannequins, and part-task training increase increased the scores of hospital nurses’ on problem solving process [ 48 ].

In the present work, the experimental group’s self-leadership scores increased after they used the program, and these scores were higher [ 49 , 50 ]. This difference can be explained by the fact that our respondents voluntarily participated in our research given their interest in self-learning programs for disaster psychological support; even in the comparison studies, participants with stronger interest in leadership education typically exhibited heightened degrees of self-leadership [ 51 ]. The increase in self-leadership scores in the current research is consistent with a previous study involving a two-hour simulation education about PPE donning and doffing, medication administration, and medical specimen treatment in a scenario of patients suspected of having infectious diseases [ 32 ]. Another research showed that simulation education on high-risk pregnancy enhances nursing students’ problem-solving processes and self-leadership [ 52 ].

Learning self-efficacy is a key variable that enables the prediction of learners’ degrees of participation in online education and the prediction of their academic achievements, as it points to the ability to manage their learning processes [ 34 , 53 ]. The results of the current research in this regard are consistent with those of a study on the online practice of basic nursing skills, which increased participants’ learning self-efficacy [ 54 ]. The researchers included an online quiz about basic nursing skills and feedback sections for learners’ self-evaluations of their performance as avenues through which to encourage autonomy in learning. A similar approach was used in the present study, which involved both a pretest for self-evaluation, direct feedback on the virtual simulation, and a self-debriefing session, enabling the participants to reflect on their simulation experiences while reviewing other participants’ answers during self-debriefing. These functions of the evaluated program were expected to factor importantly in the significant increase in the participants’ learning self-efficacy scores.

Many studies on practice education have examined participants’ motivations to transfer knowledge and skills alongside their learning self-efficacies. In the current research, the motivation to transfer scores of the experimental increased, and the difference between the two groups was statistically meaningful. This result is consistent with the findings of Park and Kweon on the simulation education about psychiatric nursing, during which post-course debriefing increased the participants’ average scores on motivation to transfer and learning self-efficacy [ 38 ]. Conversely, Kang and Kim found that a six-week simulation program for alcoholic patient care did not generate a significant increase in the participants’ motivation to transfer and learning self-efficacy scores [ 55 ]. This finding was attributed to the unfamiliarity of the local community scenario used in the research to the participants, who were in their senior year of nursing school [ 55 ]. This limitation was overcome in the current research by administering a qualitative survey of nurses’ actual demand for education on psychological support for infectious disease patients. That is, the survey presented scenarios that the participants needed.

As with other studies, the present research was encumbered by several limitations. First, the self-assessment measures used in this study may be unreliable, because they are based on individuals’ subjective perceptions and interpretations of their abilities. There is also the possibility of respondent fatigue given that the participants were compelled to answer numerous questions. Future studies should incorporate both subjective and objective measures into data collection and consider as concise an evaluation method as possible to prevent respondent fatigue. Second, this study did not establish a direct link between the obtained results and actual changes in practice or improvements in patient outcomes. We propose a follow-up study to investigate the impact of the education program examined in this study on either the mental health of patients or the quality of patient care. Third, simulation-based education tends to be accompanied with more guidance than text-based program because the former has diverse components, including quiz games, and participants are predisposed to allocate more time to simulation-based education. These may potentially influence the results. In the future, we propose to conduct research by modifying education under the same time and guided condition.

This study proposed that a well-designed virtual nursing simulation-based education program can be an effective modality with which to satisfy the educational needs of nurses in the context of infectious disease outbreaks. Such programs can be easily used by nurses anywhere and anytime before they are deployed to provide psychological support to patients with infectious diseases. They are also expected to contribute to enhancing competence in addressing disaster mental health and improving the quality of care of patients afflicted with infectious diseases.

Data availability

The datasets used and/or analyzed in this study are available from the corresponding author upon reasonable request.

Abbreviations

Coronavirus disease 2019

Randomized controlled trial

Personal protective equipment

Statistical Package for the Social Sciences

Analysis of covariance

Psychological first aid

World Health Organization. WHO coronavirus disease (COVID-19) dashboard. 2020.

Oh EG, Yu SH, Kang JH, Oh YJ, Cha KI, Jeon YH, et al. Perspectives on nursing profession for a post-COVID-19 new normal. Korean J Adult Nurs. 2020;32(3):221–2.

Article   Google Scholar  

Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. Rapid review. Lancet. 2020;395:912–20.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Çalışkan C, Özsezer G, Pay M, Demir G, Çelebi I, Koçak H. Web search behaviors and infodemic attitudes regarding COVID-19 in Turkey: a framework study for improving response and informing policy on the COVID-19 infodemic. Front Public Health. 2022;10:948478.

Article   PubMed   PubMed Central   Google Scholar  

Stangl AL, Earnshaw VA, Logie CH, Van Brakel WC, Simbayi L, Barré I, et al. The health stigma and discrimination Framework: a global, crosscutting framework to inform research, intervention development, and policy on health-related stigmas. BMC Med. 2019;17:1–13.

Tucci V, Moukaddam N, Meadows J, Shah S, Galwankar SC, Kapur GB. The forgotten plague: psychiatric manifestations of Ebola, Zika, and emerging infectious diseases. J Global Infect Dis. 2017;9(4):151.

Korea Mental Health Technology R&D Project. (2017). Korean Disaster and Mental Health Support Guidelines.

Zhang J, Wu W, Zhao X, Zhang W. Recommended psychological crisis intervention response to the 2019 novel coronavirus pneumonia outbreak in China: a model of West China Hospital. Precision Clin Med. 2020;3(1):3–8.

Liu W. The effects of virtual simulation on undergraduate nursing students’ mental health literacy: a prospective cohort study. Issues Ment Health Nurs. 2020:1–10.

Cant R, Cooper S, Sussex R, Bogossian F. What’s in a name? Clarifying the nomenclature of virtual simulation. Clin Simul Nurs. 2019;27:26–30.

Foronda C, Bauman EB. Strategies to incorporate virtual simulation in nurse education. Clin Simul Nurs. 2014;10(8):412–812.

Berman NB, Durning SJ, Fischer MR, Huwendiek S, Triola MM. The role for virtual patients in the future of medical education. Acad Med. 2016;91(9):1217–22.

Article   PubMed   Google Scholar  

Verkuyl M, Hughes M, Tsui J, Betts L, St-Amant O, Lapum JL. Virtual gaming simulation in nursing education: a focus group study. J Nurs Educ. 2017;56(5):274–80.

Cobbett S, Snelgrove-Clarke E. Virtual versus face-to-face clinical simulation in relation to student knowledge, anxiety, and self-confidence in maternal-newborn nursing: a randomized controlled trial. Nurse Educ Today. 2016;45:179–84.

Bahreini M, Shahamat S, Hayatdavoudi P, Mirzaei M. Comparison of the clinical competence of nurses working in two university hospitals in Iran. Nurs Health Sci. 2011;13(3):282–8.

Goliroshan S, Babanataj R, Aziznejadroshan P. Investigating the self-assessment of clinical competency of nurses working in Babol University of Medical Sciences hospitals. Middle East J Family Med. 2018;7(10):279.

Google Scholar  

Bam V, Diji AKA, Asante E, Lomotey AY, Adade P, Akyeampong BA. Self-assessed competencies of nurses at an emergency department in Ghana. Afr J Emerg Med. 2020;10(1):8–12.

O’Leary J. Comparison of self-assessed competence and experience among critical care nurses. J Nurs Adm Manag. 2012;20(5):607–14. https://doi.org/10.1111/j.1365-2834.2012.01394.x .

Kim KM, Choi JS. Self-perceived competency of infection control nurses based on Benner’s framework: a nationwide survey in Korea. Appl Nurs Res. 2015;28(2):175–9.

Hassankhani H, Hasanzadeh F, Powers KA, Dadash Zadeh A, Rajaie R. Clinical skills performed by Iranian emergency nurses: perceived competency levels and attitudes toward expanding professional roles. J Emerg Nurs. 2018;44(2):156–63. https://doi.org/10.1016/j.jen.2017.06.007 .

Biagioli V, Prandi C, Nyatanga B, Fida R. The role of professional competency in influencing job satisfaction and organizational citizenship behavior among palliative care nurses. J Hospice Palliat Nurs. 2018;20(4):377–84.

Alreshidi NM, Alaseeri R, Garcia M. Factors influencing absenteeism among nursing staff in the primary health care centers in hail: a preliminary study for enhancing staff commitment. Health Sci J. 2019;13(3):658–1.

Rodolfa E, Bent R, Eisman E, Nelson P, Rehm L, Ritchie P. A cube model for competency development: implications for psychology educators and regulators. Prof Psychology: Res Pract. 2005;36(4):347.

King RV, Burkle FM, Walsh LE, North CS. Competencies for disaster mental health. Curr Psychiatry Rep. 2015;17(3).

Lee JS, You S, Choi YK, Youn Hy SH. A preliminary evaluation of the training effects of a didactic and simulation-based psychological first aid program in students and school counselors in South Korea. PLoS ONE. 2017;12(7):e0181271.

Kang JY, Choi YJ. Effects of a psychological first aid simulated training for pregnant flood victims on disaster relief worker’s knowledge, competence, and self-efficacy. Appl Nurs Res. 2020:1513.

Park S, Kim S. The effects of team-based simulation education on problem solving process. Korean J Health Communication. 2019;14(2):165–72.

Australian Psychological Society. Useful skills for disaster recovery: Problem solving. 2016.

Lim A, Song Y. A scoping review of instruments for measuring problem-solving ability in simulation nursing education in Korea: a focus on process behavior survey. J Korean Acad Fundamentals Nurs. 2022;29(3):269–83.

Lee M, Lee M, Kim S. A study on nursing students’ self-leadership and their perception of learning. J Korean Acad Soc Nurs Educ. 2015;21(3):417–25. https://doi.org/10.5977/jkasne.2015.21.3.417 .

Goldsby E, Goldsby M, Neck CB, Neck CP. Under pressure: time management, self-leadership, and the nurse manager. Administrative Sci. 2020;10(3):38.

Kim JK, Song MS. Effects of respiratory infectious disease simulation-based education on nursing students of clinical competency, self-leadership and critical thinking. J Korea Academia-Industrial Cooperation Soc. 2019;20(8):93–101.

Cho GY, Seo MK. Influencing factors of learning flow, self-leadership and debriefing satisfaction on problem solving ability of nursing students in simulation learning. J Fisheries Mar Sci Educ. 2020;32(2):409–19.

Artino AR. Motivational beliefs and perceptions of instructional quality: Predicting satisfaction with online training. J Comput Assist Learn. 2008;24(3):260–70.

Yu BM, Jeon JC, Park HJ. The effects on learning motivation and self-efficacy according to the type of reflection. J Educational Inform Media. 2013;19(4):837–59.

Kim KA, Choi DW. The effect of virtual simulation in nursing education: an application of care for acute heart disease patients. J Korean Soc Simul Nurs. 2018;6(2):1–13.

Sapiano AB, Sammut R, Trapani J. The effectiveness of virtual simulation in improving student nurses’ knowledge and performance during patient deterioration: a pre and post test design. Nurse Educ Today. 2018;62:128–33.

Park SY, Kweon YR. The effect of using standardized patients in psychiatric nursing practical training for nursing college students. J Korean Acad Psychiatric Mental Health Nurs. 2012;21(1):79–88.

Ko E. Development of a virtual simulation-based nurse education program to provide psychological support for patients affected by infectious disease disasters [dissertation]. [Seoul, Korea]: Chung-Ang University.

Yoon HY, Choi YK. The development and validation of the perceived competence scale for disaster mental health workforce. Psychiatry Invest. 2019;16(11):816.

Park JW, Woo OH. The effects of PBL (problem-based learning) on problem solving process by learner’s metacognitive level. J Educational Technol. 1999;15(3):55–81.

Lee JS. The effects of process behaviors on problem-solving performance in various tests. Dissertation Abstracts International Section A: Humanities and Social Sciences. 1978;39(4-A):2149.

Manz CC. The art of self-leadership: strategies for personal effectiveness in your life and work. Prentice-Hall; 1983.44. Kim HS. The relationship between self-leadership and job satisfaction of middle school teacher [master’s thesis]. [Seoul]: Soongsil University; 2003. 83 p. 45. Ayres HW. Factors related to motivation to learn and motivation to transfer learning in a nursing population [dissertation]. [Raleigh]: North Carolina State University.

Kim HS. The relationship between self-leadership and job satisfaction of middle school teacher [master?s thesis]. [Seoul]: Soongsil University; 2003. 83 p.

Ayres HW. Factors related to motivation to learn and motivation to transfer learning in a nursing population [dissertation]. [Raleigh]: North Carolina State University.

Park H, Choi S, Choi Y, Park S, You S, Baik M, et al. Effect of Korean version of psychological first aid training program on training disaster mental health service provider. J Korean Neuropsychiatric Association. 2020;59(2):123–35.

Kim M, Kim S, Lee WS. Effects of a virtual reality simulation and a blended simulation of care for pediatric patient with asthma. Child Health Nurs Res. 2019;25(4):496–506.

Noh J, Oh EG, Kim SS, Jang YS, Chung HS, Lee O. Development and evaluation of a multimodality simulation disaster education and training program for hospital nurses. Int J Nurs Pract. 2020:e12810.

Cho JL. The effect of self-leadership and communication barriers on nursing performance in hospital nurses. J Convergence Cult Technol. 2019;5(2):239–46.

Kwon SN, Park HJ. Effects of nurses’ positive psychological capital, self-leadership, and relational bonds on organizational commitment. J Korean Acad Nurs Adm. 2020;26(3):241–50.

Yoo JY, Lee YH, Ha YK. A convergence study on the effects of self-leadership and job satisfaction on nursing performance in general hospital nurses. J Convergence Inform Technol. 2019;9(10):28–38.

Park SY, Shim CS. Effects of a simulator-based delivery education on the major satisfaction, nursing professionalism and clinical competence of the nursing students. J Korea Entertainment Ind Association (JKEIA). 2018;12(5):199.

Jeon JC. The effects on learning motivation and self-efficacy according to the type of reflection. J Educational Inform Media. 2013;19(4):837–59.

Kim S, Cho Y. Proliferation of online learning and the implications for teaching and learning in future university education. J Lifelong Learn Soc. 2018;14(4):51–78.

Kang GS, Kim Y. Development and application of integrated-simulation practice program using standardized patients: caring for alcoholism with diabetes mellitus in the community. J Korea Academia-Industrial Cooperation Soc. 2016;17(8):662–72.

Download references

Acknowledgements

The authors would like to thank Eun-Joo Choi and Dong-Hee Cho for their contributions to the development of the simulation program.

This work was supported by the National Research Foundation of Korea (NRF) through a grant funded by the Korean government (Ministry of Science and ICT) (NRF-2020R1A2B5B0100208).

Author information

Authors and affiliations.

Chung-Ang University, Red Cross College of Nursing, Seoul, South Korea

Eunjung Ko & Yun-Jung Choi

You can also search for this author in PubMed   Google Scholar

Contributions

K.E. and Y.C. wrote the design, collected data, data analysis, results, discussion, and conclusion.

Corresponding author

Correspondence to Yun-Jung Choi .

Ethics declarations

Ethics approval and consent to participate.

This study was conducted in accordance with the Declaration of Helsinki (Association World Medical, 2013) and was part of a larger study. It was approved by the Institutional Review Board of Chung-Ang University (IRB approval number 1041078-202003-HRSB-070-01CC) and retrospectively registered (21/11/2023) in the Clinical Research Information Service ( https://cris.nih.go.kr ) with trial registration number KCT0008965. All the participants provided written informed consent and were informed of the right to withdraw from participation at any time during the research until publication. Data confidentiality was ensured, and the results were provided to the participants at their request.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ko, E., Choi, YJ. Efficacy of a virtual nursing simulation-based education to provide psychological support for patients affected by infectious disease disasters: a randomized controlled trial. BMC Nurs 23 , 230 (2024). https://doi.org/10.1186/s12912-024-01901-4

Download citation

Received : 01 November 2023

Accepted : 29 March 2024

Published : 07 April 2024

DOI : https://doi.org/10.1186/s12912-024-01901-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Virtual simulation
• Psychological support
• Infectious diseases disaster
• Nursing education

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

• Resume Writing
• Resume Examples
• Cover Letter
• Remote Work
• Famous Resumes
• Try Kickresume

7 Problem Solving Skills That Aren’t Just Buzzwords (+ Resume Example)

• Julia Mlcuchova , 
• Updated April 8, 2024 9 min read

Problem-solving skills are something everybody should include on their resume, yet only a few seem to understand what these skills actually are. If you've always felt that the term "problem-solving skills" is rather vague and wanted to know more, you've come to the right place.

In this article, we're going to explain what problem-solving skills really mean. We'll talk about what makes up good problem-solving skills and give you tips on how to get better at them. You'll also find out how to make your problem-solving abilities look more impressive to those who might want to hire you.

Sounds good, right? Curious to learn more? 

In this article we’ll show you:

• What are problem solving skills;
• Why are they important; 
• Specific problem solving skills examples;
• How to develop your problem solving skills;
• And, how to showcase them on your resume.

Table of Contents

Click on a section to skip

What are problem solving skills?

Why are problem solving skills important, the best 7 problem solving skills examples, how to develop problem solving skills, problem solving skills resume example, key takeaways: problem solving skills.

First of all, they're more than just a buzzword!

Problem-solving skills are a set of specific abilities that allow you to deal with unexpected situations in the workplace, whether it be job related or team related. 

It's a complex process that involves several “sub skills” or “sub steps,” namely:

• Recognizing and identifying the issue at hand.
• Breaking the problem down into smaller parts and analyzing how they relate to one another. 
• Creating potential solutions to the problem, evaluating them and picking the best one.  
• Applying the chosen solution and assessing its outcome. 
• Learning from the whole process to deal with future problems more effectively. 

As you can see, it's not just about solving problems that are right in front of us, but also about predicting potential issues and being prepared to deal with them before they arise.  

Despite what you may believe, problem-solving skills aren't just for managers . 

Think about it this way: Why do employers hire employees in the first place? To solve problems for them!

And, as we all know, problems don't discriminate. In other words, it doesn't matter whether you're just an intern, an entry-level professional, or a seasoned veteran, you'll constantly face some kind of challenges. And the only difference is in how complex they will get.

This is also reflected in the way employers assess suitability of potential job candidates. 

In fact, research shows that the ability to deal with unexpected complications is prioritized by an overwhelming 60% of employers across all industries, making it one of the most compelling skills on your resume.

So, regardless of your job description or your career level, you're always expected to find solutions for problems, either independently or as a part of a team. 

And that's precisely what makes problem-solving skills so invaluable and universal ! 

Wondering how good is your resume?

Find out with our AI Resume Checker! Just upload your resume and see what can be improved.

As we've said before, problem-solving isn't really just one single skill. 

Instead, your ability to handle workplace issues with composure depends on several different “sub-skills”. 

So, which specific skills make an employee desirable even for the most demanding of recruiters? 

In no particular order, you should focus on these 7 skills : 

• Analytical skills
• Research skills
• Critical thinking 
• Decision-making
• Collaboration
• Having a growth mindset

Let's have a look at each of them in greater detail!

#1 Analytical skills

Firstly, to truly understand complex problems, you need to break them down into more manageable parts . Then, you observe them closely and ask yourself: “ Which parts work and which don't,” How do these parts contribute to the problem as a whole,” and "What exactly needs to be fixed?” In other words, you gather data , you study it, and compare it - all to pinpoint the cause of the issue as closely as possible.

#2 Research skills

Another priceless tool is your research skills (sometimes relying on just one source of information isn't enough). Besides, to make a truly informed decision , you'll have to dig a little deeper. Being a good researcher means looking for potential solutions to a problem in a wider context. For example: going through team reports, customer feedback, quarterly sales or current market trends.  

#3 Critical thinking

Every employer wants to hire people who can think critically. Yet, the ability to evaluate situations objectively and from different perspectives , is actually pretty hard to come by. But as long as you stay open-minded, inquisitive, and with a healthy dose of skepticism, you'll be able to assess situations based on facts and evidence more successfully. Plus, critical thinking comes in especially handy when you need to examine your own actions and processes. 

 #4 Creativity

Instead of following the old established processes that don't work anymore, you should feel comfortable thinking outside the box. The thing is, problems have a nasty habit of popping up unexpectedly and rapidly. And sometimes, you have to get creative in order to solve them fast. Especially those that have no precedence. But this requires a blend of intuition, industry knowledge, and quick thinking - a truly rare combination. 

#5 Decision-making

The analysis, research, and brainstorming are done. Now, you need to look at the possible solutions, and make the final decision (informed, of course). And not only that, you also have to stand by it ! Because once the train gets moving, there's no room for second guessing. Also, keep in mind that you need to be prepared to take responsibility for all decisions you make. That's no small feat! 

#6 Collaboration

Not every problem you encounter can be solved by yourself alone. And this is especially true when it comes to complex projects. So, being able to actively listen to your colleagues, take their ideas into account, and being respectful of their opinions enables you to solve problems together. Because every individual can offer a unique perspective and skill set. Yes, democracy is hard, but at the end of the day, it's teamwork that makes the corporate world go round. 

#7 Having a growth mindset

Let's be honest, no one wants their work to be riddled with problems. But facing constant challenges and changes is inevitable. And that can be scary! However, when you're able to see these situations as opportunities to grow instead of issues that hold you back, your problem solving skills reach new heights. And the employers know that too!

Now that we've shown you the value problem-solving skills can add to your resume, let's ask the all-important question: “How can I learn them?”

Well…you can't. At least not in the traditional sense of the word. 

Let us explain: Since problem-solving skills fall under the umbrella of soft skills , they can't be taught through formal education, unlike computer skills for example. There's no university course that you can take and graduate as a professional problem solver. 

But, just like other interpersonal skills, they can be nurtured and refined over time through practice and experience. 

Unfortunately, there's no one-size-fits-all approach, but the following tips can offer you inspiration on how to improve your problem solving skills:

• Cultivate a growth mindset. Remember what we've said before? Your attitude towards obstacles is the first step to unlocking your problem-solving potential. 
• Gain further knowledge in your specialized field. Secondly, it's a good idea to delve a little deeper into your chosen profession. Because the more you read on a subject, the easier it becomes to spot certain patterns and relations.  
• Start with small steps. Don't attack the big questions straight away — you'll only set yourself up for failure. Instead, start with more straightforward tasks and work your way up to more complex problems. 
• Break problems down into more digestible pieces. Complex issues are made up of smaller problems. And those can be further divided into even smaller problems, and so on. Until you're left with only the basics. 
• Don't settle for a single solution. Instead, keep on exploring other possible answers.
• Accept failure as a part of the learning process. Finally, don't let your failures discourage you. After all, you're bound to misstep a couple of times before you find your footing. Just keep on practicing. 

How to improve problem solving skills with online courses

While it’s true that formal education won’t turn you into a master problem solver, you can still hone your skills with courses and certifications offered by online learning platforms :

• Analytical skills. You can sharpen your analytical skills with Data Analytics Basics for Everyone from IBM provided by edX (Free); or Decision Making and Analytical Thinking: Fortune 500 provided by Udemy (＄21,74).
• Creativity. And, to unlock your inner creative mind, you can try Creative Thinking: Techniques and Tools for Success from the Imperial College London provided by Coursera (Free).
• Critical thinking. Try Introduction to Logic and Critical Thinking Specialization from Duke University provided by Coursera (Free); or Logical and Critical Thinking offered by The University of Auckland via FutureLearn.  
• Decision-making. Or, you can learn how to become more confident when it's time to make a decision with Decision-Making Strategies and Executive Decision-Making both offered by LinkedIn Learning (1 month free trial).
• Communication skills . Lastly, to improve your collaborative skills, check out Communicating for Influence and Impact online at University of Cambridge. 

The fact that everybody and their grandmothers put “ problem-solving skills ” on their CVs has turned the phrase into a cliche. 

But there's a way to incorporate these skills into your resume without sounding pretentious and empty. Below, we've prepared a mock-up resume that manages to do just that.

FYI, if you like this design, you can use the template to create your very own resume. Just click the red button and fill in your information (or let the AI do it for you).

Problem solving skills on resume example

This resume was written by our experienced resume writers specifically for this profession.

Why this example works?

• Firstly, the job description itself is neatly organized into bullet points .  
• Instead of simply listing soft skills in a skills section , you can incorporate them into the description of your work experience entry.  
• Also, the language here isn't vague . This resume puts each problem-solving skill into a real-life context by detailing specific situations and obstacles. 
• And, to highlight the impact of each skill on your previous job position, we recommend quantifying your results whenever possible. 
• Finally, starting each bullet point with an action verb (in bold) makes you look more dynamic and proactive.

To sum it all up, problem-solving skills continue gaining popularity among employers and employees alike. And for a good reason!

Because of them, you can overcome any obstacles that stand in the way of your professional life more efficiently and systematically. 

In essence, problem-solving skills refer to the ability to recognize a challenge, identify its root cause, think of possible solutions , and then implement the most effective one. 

Believing that these skills are all the same would be a serious misconception. In reality, this term encompasses a variety of different abilities , including:

In short, understanding, developing, and showcasing these skills, can greatly boost your chances at getting noticed by the hiring managers. So, don't hesitate and start working on your problem-solving skills right now!

Julia has recently joined Kickresume as a career writer. From helping people with their English to get admitted to the uni of their dreams to advising them on how to succeed in the job market. It would seem that her career is on a steadfast trajectory. Julia holds a degree in Anglophone studies from Metropolitan University in Prague, where she also resides. Apart from creative writing and languages, she takes a keen interest in literature and theatre.

Related Posts

10 buzzwords you should stop using on your resume, how to write the perfect “no experience” resume.

• 11 min read

Share this article

Join our newsletter.

Every month, we’ll send you resume advice, job search tips, career hacks and more in pithy, bite-sized chunks. Sounds good?

Read our research on: Gun Policy | International Conflict | Election 2024

Regions & Countries

About half of americans say public k-12 education is going in the wrong direction.

About half of U.S. adults (51%) say the country’s public K-12 education system is generally going in the wrong direction. A far smaller share (16%) say it’s going in the right direction, and about a third (32%) are not sure, according to a Pew Research Center survey conducted in November 2023.

Pew Research Center conducted this analysis to understand how Americans view the K-12 public education system. We surveyed 5,029 U.S. adults from Nov. 9 to Nov. 16, 2023.

The survey was conducted by Ipsos for Pew Research Center on the Ipsos KnowledgePanel Omnibus. The KnowledgePanel is a probability-based web panel recruited primarily through national, random sampling of residential addresses. The survey is weighted by gender, age, race, ethnicity, education, income and other categories.

Here are the questions used for this analysis , along with responses, and the survey methodology .

A majority of those who say it’s headed in the wrong direction say a major reason is that schools are not spending enough time on core academic subjects.

These findings come amid debates about what is taught in schools , as well as concerns about school budget cuts and students falling behind academically.

Related: Race and LGBTQ Issues in K-12 Schools

Republicans are more likely than Democrats to say the public K-12 education system is going in the wrong direction. About two-thirds of Republicans and Republican-leaning independents (65%) say this, compared with 40% of Democrats and Democratic leaners. In turn, 23% of Democrats and 10% of Republicans say it’s headed in the right direction.

Among Republicans, conservatives are the most likely to say public education is headed in the wrong direction: 75% say this, compared with 52% of moderate or liberal Republicans. There are no significant differences among Democrats by ideology.

Similar shares of K-12 parents and adults who don’t have a child in K-12 schools say the system is going in the wrong direction.

A separate Center survey of public K-12 teachers found that 82% think the overall state of public K-12 education has gotten worse in the past five years. And many teachers are pessimistic about the future.

Related: What’s It Like To Be A Teacher in America Today?

Why do Americans think public K-12 education is going in the wrong direction?

We asked adults who say the public education system is going in the wrong direction why that might be. About half or more say the following are major reasons:

• Schools not spending enough time on core academic subjects, like reading, math, science and social studies (69%)
• Teachers bringing their personal political and social views into the classroom (54%)
• Schools not having the funding and resources they need (52%)

About a quarter (26%) say a major reason is that parents have too much influence in decisions about what schools are teaching.

How views vary by party

Americans in each party point to different reasons why public education is headed in the wrong direction.

Republicans are more likely than Democrats to say major reasons are:

• A lack of focus on core academic subjects (79% vs. 55%)
• Teachers bringing their personal views into the classroom (76% vs. 23%)

In turn, Democrats are more likely than Republicans to point to:

• Insufficient school funding and resources (78% vs. 33%)
• Parents having too much say in what schools are teaching (46% vs. 13%)

Views also vary within each party by ideology.

Among Republicans, conservatives are particularly likely to cite a lack of focus on core academic subjects and teachers bringing their personal views into the classroom.

Among Democrats, liberals are especially likely to cite schools lacking resources and parents having too much say in the curriculum.

Note: Here are the questions used for this analysis , along with responses, and the survey methodology .

Sign up for our weekly newsletter

Fresh data delivered Saturday mornings

‘Back to school’ means anytime from late July to after Labor Day, depending on where in the U.S. you live

Among many u.s. children, reading for fun has become less common, federal data shows, most european students learn english in school, for u.s. teens today, summer means more schooling and less leisure time than in the past, about one-in-six u.s. teachers work second jobs – and not just in the summer, most popular.

About Pew Research Center Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of The Pew Charitable Trusts .