Measuring Student Success Skills – Critical Thinking, Part 1: Design

We are on a new research and evidence journey at PBLWorks, creating tools and resources that partners can use to capture and share the positive effects of PBL with key stakeholders in their school and district communities.

In collaboration with the Center for Assessment at the National Center for the Improvement of Educational Assessment (NCIEA), we are designing rubrics to measure success skills developed through PBL. The first stop on our journey is critical thinking.

This post, the first in a series, summarizes a recent literature review on critical thinking conducted by the Center for Assessment,¹ highlighting key research findings from hundreds of studies examining critical thinking. This post focuses on designing a project for critical thinking. In the second post, we will share research insights to help guide the teaching of critical thinking in PBL. Our third post will focus on measuring critical thinking in projects. 

What is critical thinking? 

In the last 30 years, educators, scholars, and leading experts in success skills (i.e.,  deeper learning or 21st century competencies) have come to a consensus definition of critical thinking as a cognitive competency:^{1 2}  While critical thinking is a close cousin to problem solving, decision making, and creative thinking, it is a distinct cognitive process.”³

Critical thinking is judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the considerations upon which that judgment is based…The ideal critical thinker is habitually inquisitive, well-informed, trustful of reason, open-minded, flexible, fair-minded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider…and persistent in seeking results which are as precise as the subject and the circumstances of inquiry permit.¹

Taxonomy of Deeper Learning Domains & Competencies ⁴

Critical thinking describes a process with two aspects:

• The ability to think critically
• The disposition to apply critical thinking skills 

In other words, a student may know how to evaluate the quality of evidence for or against a given claim (ability), but decide not to apply that cognitive skill in a particular situation (disposition). In such situations, critical thinking has not taken place. The ability to think critically must converge with the prevailing tendency or disposition to do so. 

For example, PBLWorks National Faculty member Erin Brandvold asks students to put a historic revolution on trial to decide whether it effectively helped the population of a country in “The Revolutions Project.” In the project, she teaches the skills of utilizing primary and secondary sources to create an argument based on evidence (ability). Her students then apply the skills learned during the project in a final mock trial where they debate against the opposing viewpoint (disposition). Her students are motivated to apply argumentative skills in a real-world scenario (a trial). Find out more about her project in this video detailing the project's implementation in her classroom. 

How is critical thinking developed?

Here are four important findings from research about how critical thinking is developed:

1. Critical thinking begins at a very young age and can be developed at any age.⁵ There is no evidence to suggest that there is a single age when children are developmentally ready to learn complex ways of thinking.⁶ Further, there is no research-based developmental progression for critical thinking.


2. A mixed instructional approach is the most effective way to develop critical thinking skills and dispositions; one that combines both integrating critical skills development into subject matter content and providing explicit instruction on critical thinking.⁷


3. Dialogue and discussion in collaborative groups and authentic or real-life problems are two effective instructional strategies that support the development of critical thinking.⁷


4. Students need opportunities to see critical thinking skills, such as analysis and inference, explicitly modeled and opportunities to apply those skills in discipline-specific work. Simply telling students to analyze a text is not enough. Students need to be taught what it means to analyze including how to examine ideas, identify arguments, and decode argument structures. 

How do I design a Gold Standard Project focused on critical thinking? 

If you have not yet checked out myPBLWorks.org, you will want to register. In our recently-launched online PBL platform, we are building a suite of tools and resources to support you as you design and facilitate Gold Standard Projects. 

1. Start by identifying key knowledge and understanding (i.e., standards) and critical thinking as your focus success skill. Identifying the learning targets is central to the design of a high-quality project aligned to Gold Standard Project Design Elements. You may choose to focus on one dimension of critical thinking (such as evaluation) or on multiple dimensions of critical thinking in your project. 


2. Create a driving question that promotes critical thinking. You can do this by framing your question for evaluation or judgment. For evaluation, craft a driving question using “best” or “most effective?” For example, “What is the most effective way to reduce carbon emissions?” You can also end with “so that…” to get at evaluation: How can we create a new school library design so that students can collaborate on projects?” For judgment, craft a question that requires students to use evidence to form an opinion: “Was the war worth it?”⁸


3. Include authenticity in your project to promote critical thinking. If a student is asked to demonstrate critical thinking (ability) within a real-world context, they will be more likely to apply what they know within this relevant learning experience (disposition). What does it take for a project to be authentic? Find out here. 


4. Use our critical thinking rubrics for grades K-2, 3-5, and 6-12 to design your project. They provide explicit guidance for designing each of the project phases: 
   • Launching the Project
   • Building Knowledge, Understanding, and Skills
   • Developing and Revising Ideas and Products 
   • Presenting Products and Answers to Driving Question


5. Use the research evidence on critical thinking in the literature review³ to guide your project design. For example, (a) include intellectually challenging dialogue in collaborative groups and (b) make sure the project authentically promotes real-life transfer of learning to new or novel contexts.   

End Notes 

¹ Facione, P.A. (1990). critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Newark, DE: American Philosophical Association, p. 3. https://files.eric.ed.gov/fulltext/ED315423.pdf

² National Research Council. (2012). Education for life and work: developing transferable knowledge and skills in the 21st century. (J. W. Pellegrino & M. L. Hilton, Eds.). Washington, DC: National Academies Press. https://www.nap.edu/catalog/13398/education-for-life-and-work-developing-transferable-knowledge-and-skills

³ Evans, C. & Marion, S. (2019). Measuring Student Success Skills: A Review of the Literature on critical thinking Including Implications for Rubric Design and Use. Dover, NH: National Center for the Improvement of Educational Assessment. https://www.nciea.org/library 

⁴ Condliffe, B., Quint, J. Visher, M. G., Bangser, M. R., Drohojowska, S. Saco, L., Nelson, E. (2017). Project Based Learning: A Literature Review – Working Paper. Oakland, CA: MDRC. https://www.mdrc.org/sites/default/files/Project-Based_Learning-LitRev_Final.pdf 

⁵ Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, C. A., & Persson, T. (2015). Strategies for teaching students to think critically: A meta-analysis. Review of Educational Research, 85(2), 275–314. https://doi.org/10.3102/0034654314551063

⁶ Silva, E. (2008). Measuring skills for the 21st century. Washington, DC: Education Sector.
Retrieved from http://elenamsilva.com/wp-content/uploads/2013/05/MeasuringSkills.pdf

⁷ Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78(4), 1102–1134. https://doi.org/10.3102/0034654308326084

⁸ Boss. S. (2013). PBL for 21st Century Success: Teaching Critical Thinking, Collaboration, Communication, and Creativity.  Novato, CA: Buck Institute for Education.