carol gittens, phd santa clara university wasc assessment leadership academy january 26, 2012
TRANSCRIPT
![Page 1: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/1.jpg)
Critical Thinking: What It Is &
How To Assess It
Carol Gittens, PhDSanta Clara University
WASC Assessment Leadership Academy
January 26, 2012
![Page 2: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/2.jpg)
Goals for the SessionAugment your understanding of the
definition of critical thinking
Expand your repertoire of classroom, program, and institution level critical thinking assessment strategies
Engage and Affirm your critical thinking skills and positive critical thinking habits of mind
![Page 3: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/3.jpg)
Challenging a Few MythsCT is what you learn at school (but it
doesn’t apply to real life).
Critical thinking naturally improves just
from being in college.
Nobody knows what “critical thinking”
means.
Whatever it is, CT can’t be measured…
© 2011 Measured Reasons, Hermosa Beach, CA
![Page 4: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/4.jpg)
Failures of critical thinking contribute to…patient deaths * lost revenue * ineffective
law enforcement * job loss * gullible voters * garbled communications * imprisonment * combat casualties * upside down mortgages * vehicular homicide * bad decisions * unplanned pregnancies * financial mismanagement * heart disease * family violence * repeated suicide attempts * divorce * drug addiction * academic failure * … * … *
© 2011 Peter A. & Noreen C. Facione and Measured Reasons, Hermosa Beach, CA
WHAT WERE WE THINKING?
![Page 5: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/5.jpg)
Novel Question Human Reflective Response Time
Tick
TickTick
Tick
Tick
Tick
Humans need 11 - 16 secondsto process a novel question.
Tick
Tick
TickTick
TickTick
TickTick
Tick
What does this mean for a student faced with an novel question in a classroom, field, or testing
setting?
And what should the instructor, supervisor or assessor do?
in contexts of uncertainty, risk
![Page 6: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/6.jpg)
“Critical Thinking” - Working Consensus
Purposeful, reflective judgment which manifests itself in reasoned consideration
of evidence, context, methods, standards, and conceptualizations
in deciding what to believe or what to
do.The Delphi Report: Executive Summary: (1990), The California Academic Press, or ERIC Doc ED315 423
![Page 7: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/7.jpg)
Purposeful, reflective judgment which manifests itself in reasoned consideration of evidence, context, methods,
standards, and conceptualizations in deciding what to believe or what to do.
![Page 8: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/8.jpg)
Three Ways Humans Reason Bottom-Up:
Observations Hypotheses Scientific Thinking
Top-Down: Beliefs & Values Applications Ideological Thinking
This Is Like That: Familiar Unfamiliar Analogical / Scripted Thinking
© 2011 Measured Reasons LLC, Hermosa Beach, CA. All rights reserved
![Page 9: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/9.jpg)
Critical Thinking Habits of Mind
(
American Philosophical Association Delphi Report: Consensus Definition of Critical Thinking (1990) Download : CT Resources Link www.insightassessment.com
CCTDICalifornia Critical Thinking Dispositions InventoryAvailable in 20 culturally competent translations
![Page 10: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/10.jpg)
Positive Habit Negative Habit
© 2011 Measured Reasons, Hermosa Beach, CA. All rights reserved.
![Page 11: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/11.jpg)
© 2011 Measured Reasons, Hermosa Beach, CA
![Page 12: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/12.jpg)
Planning Design Measurement Interpretation Value
Assessment Questions
![Page 13: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/13.jpg)
1. How is assessment different than grading?
2. Is our main goal developmental or summative?
3. Which curricular level should we assess? lessoncoursegeneral education requirementmajordegree programtotal institutional experience
Assessment Planning Questions:
![Page 14: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/14.jpg)
4. If we hope to show positive growth, do we have clean baseline / pretest data?
5. If we hope to show comparative strength, do we have relevant benchmark / norms?
6. If we hope to demonstrate accomplishing goals, do we a clear criteria of success?
Design Questions:
![Page 15: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/15.jpg)
7. Do we have a focused, accurate, consensus definition of the construct we seek to measure?
8. Do we have one or more valid, reliable ways of gathering data about that construct?
9. Do we have a clear idea of which students we shall need to sample?
10. Must we gather data from them all, or can we use representative sampling?
Measurement Questions:
© 2012 Measured Reasons, Hermosa Beach, CA
![Page 16: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/16.jpg)
Three Basic Options for Measuring Learning Outcomes
www.InsightAssessment.com Measuring Critical Thinking Worldwide
1. Rubrics and Rating ToolsQualitative Rating Forms, Typological Matches, ChecklistsRequire practiced judgment and inter-rater calibration
Adaptable to performance and written data
3. Testing InstrumentsBaseline / Cross-Sectional / Longitudinal
Time thrifty / Norm referencedPotential for comparisons & data integration
2. Self ReportsJournals, Self Critiques, Focus Groups, QuestionnairesInsights about personal progress and deficiency Require significant resources for data analysis
Are we consistently getting a valid and reliable measure of the phenomenon we intended to target?
![Page 17: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/17.jpg)
11. When and where should we gather data?
12. How can we motivate student to give their best effort?
13. Who will collect and store the data?
14. Who will score, rate, compile and analyze the data?
Logistical & Tactical Questions:
© 2012 Measured Reasons, Hermosa Beach, CA
![Page 18: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/18.jpg)
Teaching and Assessment Tool Scoring Rubrics Holistic Critical Thinking
Scoring Rubric (HCTSR)
Excellent/Strong = 4
Adequate/Satisfactory = 3
Deficient/Weak = 2
Truly Dreadful = 1
www.InsightAssessment.com Measuring Critical Thinking Worldwide
Describe three or four levels of performance
![Page 19: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/19.jpg)
© 2010 Peter A. & Noreen C. Facione and Measured Reasons, Hermosa Beach, CA
Share a scoring rubric from day-1
to establish expectations
to make “critical thinking” operational for students
![Page 20: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/20.jpg)
Inductive Reasoning Deductive Reasoning Analysis & Interpretation Inference Explanation & Evaluation
Total Percentile Demographics
Scale Scores on the CCTST
© 2011 Measured Reasons, Hermosa Beach, CA. All rights reserved.
![Page 21: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/21.jpg)
ANYWHERE U., May 2010 – CCTST Raw Score – Qualitative Categories
© 2011 Measured Reasons, Hermosa Beach, CA. All rights reserved.
Weak
Moderate
Strong
![Page 22: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/22.jpg)
Insight Assessment LLCMeasuring Critical Thinking Worldwide
Publishers of the California Critical Thinking Tests& other measures of reasoning and problem
solving: California Critical Thinking Skills Test (CCTST) Test of Everyday Reasoning (TER) California Critical Thinking Disposition Inventory (CCTDI) Health Sciences Reasoning Test (HSRT – Parts 1&2 ) California Measure of Mental Motivation (CM3) Business Critical Thinking Skills Test (BCTST) Business Reasoning Test (BRT) Business Attitude Inventory (BAI) Legal Studies Reasoning Profile (LSRP - Parts 1 & 2) Military and Defense Critical Thinking Inventory (MDCTI -Parts 1&2) Quant Q (Test of Mathematical Reasoning)
Willing and Able
![Page 23: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/23.jpg)
Interpretation Question: 15. By exit, what percentage of students should be in a mid-range or higher category?
35 18
180 MBA Students
127
© 2012 Measured Reasons, Hermosa Beach, CA
Using Quantitative / Categorical
Scores
![Page 24: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/24.jpg)
2005 2006 2007 2008 2009 20100
5
10
15
20
25
30
35
40
PretestPosttest
Interpretation Question 16: Should we benchmark internally to our own past performance or to norms of actual or aspirational peers?
![Page 25: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/25.jpg)
17. Do cross-sectional comparisons mean anything, or are the differences explainable by student attrition, maturation, selection bias, etc.?
18. What level of achievement should we have expected of our students as a group?
19. If we see gains, are they educationally significant and for which subgroups of students?
More Interpretive Questions:
© 2012, and Measured Reasons, Hermosa Beach, CA
![Page 26: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/26.jpg)
TRUTH-SEEKING:
the courageous desire for the best possible
knowledge in any given context,
the inclination to ask hard questions,
and to follow reason and evidencewherever they lead.
© 2011 Measured Reasons, Hermosa Beach, CA
![Page 27: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/27.jpg)
Drs. Peter & Noreen Facione – © 2007 – www.insightassessment.com
Truth-seeking: A Profile of 155 Entering Freshmen 1992
10 15 20 25 30 35 40 45 50 55 60
Mean = 35.7s.d. = 5.6
Weak dispositionToward truth-seeking
Strong dispositionToward truth-seeking
A Look across Four Years at the Disposition toward Critical Thinking Among Undergraduate Students, C. Giancarlo and P. Facione, The Journal of General Education, (2001). Volume 50, number 1. 29-55.
![Page 28: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/28.jpg)
Drs. Peter & Noreen Facione – © 2007 – www.insightassessment.com
Truth-seeking: Exiting Seniors 1996The same 155 students 4 years later
10 15 20 25 30 35 40 45 50 55 60
Positive movement across categories 38.8%
Negative movement across categories 10.9%
Mean = 38.5s.d. = 6.2
A Look across Four Years at the Disposition toward Critical Thinking Among Undergraduate Students, C. Giancarlo and P. Facione, The Journal of General Education, (2001). Volume 50, number 1. 29-55.
Strong dispositionToward truth-seeking
Weak dispositionToward truth-seeking
![Page 29: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/29.jpg)
Gains: Critical Thinking Disposition
www.InsightAssessment.com Measuring Critical Thinking Worldwide
![Page 30: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/30.jpg)
Poor CT Skills Moderate CT Skills Strong CT Skills
Critical Thinking Scores – Implications for Learning
Avers
e
N
eg
ati
ve
Am
biv
ale
nt
Posit
ive
Str
on
g
© 2008 Insight Assessment, The California Academic Press LLC: Millbrae, CA.
60
50
40
30
20
10
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34
Name New Transfers Students Date ___Fall 2008 Pre-Test_____Skills Measure ___CCTST_- 2000_ Dispositions Measure ____CCTDI__
Strong Dispositions / Strong Skills
Negative Dispositions / Poor Skills
Neutral Dispositions / Moderate Skills
Strong Dispositions / Moderate Skills
![Page 31: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/31.jpg)
Use fair-minded, reflective thinking when interpreting
results! Use Powerful Critical Thinking Skills:
Interpret the data display Analyze and explain what you find What can we infer from these data? Evaluate the inference we just drew Rethink a judgment in light of new facts
Call Forth Positive CT Habits of Mind: Go ahead, Ask. Have courage and seek truth Follow the data and reasons wherever they lead Keep an open-mind about what others have to say Proceed systematically, don’t jump to conclusions Don’t lock yourself in – be ready to reconsider when
conditions change
![Page 32: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/32.jpg)
20. How do we fund the faculty time, staff positions, and operating budget, and how do we manage impact of outcomes assessment on other academic and institutional priorities?
21. Which unit has the expertise and objectivity to analyze, interpret and report on the data we gather?
22. How can we maximize the benefits of outcomes assessment to inform and support other functions?
Accreditation, Admissions, Student Success, Institutional Accountability, Curricular Development, Research, Grants, Fund-Raising
Value Questions:
© 2012, and Measured Reasons, Hermosa Beach, CA
![Page 33: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/33.jpg)
Assessing Critical Thinking
Are there a variety of assessments that engage students in one or more critical thinking skills?
Are there assessments that elicit students’ critical thinking habits of mind?
Do the assessments provide opportunities to evaluate students’ independent critical thinking and their thinking in groups?
Are there a sufficient number of assessments that will be reviewed and returned to students so that they receive frequent feedback on their performances?
What benchmarking data will be used for CT assessments at the course, program, and institutional levels?
How will assessment results be shared to the campus community so that students are able to practice their critical thinking skills and dispositions and make appropriate modifications?
© 2011 Measured Reasons, Hermosa Beach, CA. All rights reserved.
![Page 34: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/34.jpg)
Thoughts on valid and reliable methods to demonstrate gains on desired outcomes
Use local talent and good data
Student motivation and timing
Data analysts and true measures
www.InsightAssessment.com Measuring Critical Thinking Worldwide
Use the right tools
Attention to design is needed
Correct calibration and clear interpretation
![Page 35: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/35.jpg)
Use thinking verbs to express outcomes and to ask questions in class and on assessment tools.
ParticipateListenKnowUnderstandRecognize
IdentifyCategorizeNameDiscuss
AnalyzeInterpretExplainInferEvaluate
To measure a phenomenon it must first be present
© 2011 Measured Reasons, Hermosa Beach, CA
![Page 36: Carol Gittens, PhD Santa Clara University WASC Assessment Leadership Academy January 26, 2012](https://reader035.vdocuments.us/reader035/viewer/2022081511/56649dd25503460f94ac851e/html5/thumbnails/36.jpg)
Why Teach and Assess for Critical Thinking?
In education measure what you value because you get what you measure.
Critical thinking – purposeful reflective judgment - is the key to academic success, a necessary element in every professional endeavor, and a central factor in individual and communal adaptation and survival.
© 2011 Peter A. & Noreen C. Facione and Measured Reasons, Hermosa Beach, CA