Dr. Hari Singh

School of Business

Humboldt State University

*Using Rubrics to Analyze Critical

Thinking

Creating a Stackable set of skills

• Discuss history and uses or rubrics.

• Show how rubrics can reduce a variety of problems.

• Creating a roadmap that develops consensus among faculty, reduce student complaints, and can be a basis for assessment.

• For critical thinking rubrics can be made progressively more challenging.

• Advance skills require not only evaluation of information but synthesis of knowledge, and development of advanced research skills.

*Using Rubrics for skills

• A rubric is a mechanism or tool for developing an evaluation scale across different dimensions.

• Most rubrics are analytical: they decompose a skill or subject into key dimensions or traits.

• Each component can be evaluated separately.

• Rubrics are criticized for resulting in excessive rigidity or inhibiting innovation.

• A rubric is a tool and it can obviously be misused.

• The important thing is to ensure that the tool is effective and flexible.

*What is a rubric?

• In the early part of the twentieth century, rubrics were constructed to develop writing scales.

• Objective measurement of student writing on some scale that could by applied uniformly across classes and instructors.

• Top Down: Earlier rubrics were developed typically at universities and applied by school administrators.

• The goal was trying to ensure more consistency and sometimes resulted in excessive standardization.

*Some History of Rubrics

*Relevance of rubrics to Learning

*Historical development of rubrics violated the “the law of distal diminishment”.

*Any tool developed for effective teaching becomes less useful (potentially more harmful) if it originates far away from the class room.

*Students and teachers should provide input on a continuous basis for the construction and modification of rubrics.

*An iterative process: Experience from the classroom provides relevant feedback for developing a more nuanced rubric.

Rubric should not become rigid straight jackets that reduce flexibility and creativity.

A rubric is merely a classifying device.

The key issue is what is classified and what type of consistency is derived from it.

The consistency should be about the performance thresholds of the student learning outcomes.

* Descriptive not prescriptive

• Performance thresholds should allow flexibility in different pedagogical styles, different teaching delivery methods, and innovative ways encouraging student learning.

• Striving for consistency in evaluating student learning outcomes is fully consistent with allowing and encouraging different teaching styles and methods.

*Flexibility is key

• Lack of consensus among faculty about the specific types of performance thresholds.

• Lack of consistency of academic standards for these performance thresholds across Classes.

• Inadequate transparency of grading students for different thresholds.

• More student complaints because threshold performance level is not transparent and communicated clearly to students.

• Difficulty in assessing student learning outcomes (SLO) consistently.

*Existing problems

Constructing a rubric encourages teachers to develop a consensus about the key learning goals of the course.

As Callison (2000; P. 31) states, “Rubrics are texts that are the visible signs of agreed upon values. They cannot contain all the nuances of the evaluation community’s values, but they do contain the central expression of those values.”

Facilitate communication between students, teachers and other stakeholders by providing a context for these discussions to take place.

Callison, D. (2000). Rubrics. School Library Media Activities Monthly. 17 (2), 34-36, 42.

* Consensus on learning values

Popham (2003) points out that a well constructed rubric can help students to understand more clearly the expectations of instructors.

The transparency that rubrics provide for ranking, ratings, and grades can make the teaching process more meaningful for students

Provide a set of stackable skills that can be made progressively more difficult.Greater transparency will result in less complains by students .

Popham, W.J. (2003). Test better, teach better: The instructional role of assessment. Alexandria, VA: Association for supervision and curriculum development.

 

*Transparency for students

Bresciani et al. (2004) point out that well developed rubrics can serve as effective and transparent norms to bring expectations of faculty and students for a consistent vision.

Increases consistency in class standards so that similar student expectations are applied across different classes of the same course.  

Rubrics can be progressively benchmarked at higher levels.

Progressively increase the challenge and difficulty of critical thinking in subsequent courses so that complex learning skills can be mastered.

Rubrics can provide a transparent and clear roadmap for a ratcheting up the standards across a sequence of courses.

Bresciani, M.J., Zelna, C.L., & Anderson, J.A. (2004) Assessing student learning and development: A Handbook for practitioners. National Association of Student personnel Administrators.

* Transparent & Progressive norms

The consistency of learning goals and a detailed roadmap of how to assess the goals is a key issue.

It reduces the subjective component of grading (Moskal, 2000).

Rubrics help identify specific weak points to strengthen the overall chain of learning.

Moskal, B.M. (2000) Scoring rubrics: What, when and how? Practical Assessment research and Evaluation, 7 (3).

 

*Facilitating assessment

 

Time line should allow a pilot run.

Find champions among faulty for classes where goals can be tested.

Bad results don’t reflect on instructors.

Typically assessment should be towards the end of the program.

However, a pre-test at the onset of the program is useful.

Form an Assessment Group which includes all disciplines\skill areas.

*How to Set up the process

Students perceptions about learning are less reliable.

Student outcomes of learning can be based on class assignments.

Benchmarking learning thresholds is useful.

Instructors, students, and assessors need a common consensus.

A well constructed rubric is a good vehicle for achieving clarity and consensus among all groups.

*Insights about methods

• Identifying 5 or 6 dimensions to be tested (use student feedback)

• Deciding 5 thresholds for each dimension (use student feedback)

• How to scale each dimension?

• Examples of rubrics that work well

• Discussion among participants with rubric examples

*Constructing Rubrics