Using Rubrics for Assessment: A Primer

Marcel S. KerrSummer 2007

Presentation Purpose Educate faculty on uses of rubrics Preparation for General Education

Assessment Plan Learn to write rubrics for multiple

purposes Student learning assessment Course assessment Program assessment

Presentation Objectives Define rubric Differentiate types of rubrics Identify uses of rubrics List advantages of using rubrics Construct a rubric Determine validity & reliability End-of-training Quiz Complete Rubric Training Rubric

What are Rubrics?

A rubric is a scoring scale used to assess performance along a task-specific set of criteria rather than a single numeric score.

A means of formative assessment Purposes of rubrics:

Describe what “outstanding” or “satisfactory” or “inadequate” performance looks like

Articulate the criteria against which student learning is judged

Provide an objective guide for assessment

What do Rubrics Assess? Assessment level:

Student learning Courses Programs Institutions

A stated learning objective: Content – the knowledge specific to a given

discipline Performance – skills and internal processes

required for achievement in a given discipline

Types of Rubrics Analytic – are more specific & easier

to apply; break down the objective into dimensions (criteria) and judge them individually Used when large number of criteria;

criteria are weighted Holistic – are more general; assess

the entire objective on one scale Used for quick or minor assessments;

open-ended questions; skills

Rubric Basic Structure There are four basic features:

Stated Objective or Purpose - title Scoring Criteria – characteristics of

good performance on the task; Levels of Performance – defined

degrees of competency Descriptors – spell out briefly what is

expected at each level of performance

Rubric Basic Structure

Criteria 1 2 3Number of Sources

1-4 5-9 10-12

Historical Accuracy

Lots of historical inaccuracies

Few inaccuracies No apparent inaccuracies

Organization Cannot tell from which source information came

Can tell with some difficulty where information came

Can easily tell from which sources information was drawn

Use of APA Format

Lots of APA errors Few APA errors No apparent APA errors

Objective: Research Paper

Uses of Rubrics Set evaluator & performer expectations

Criteria by which work is judged

Difference between excellent & weak work

Formative student feedback

Grade assignments

Standardize grading across graders

Assess programs (GEC)

Rubrics for Program

Assessment Rationale for use:

Faculty created, owned, and operated Low cost & reusable National tests are expensive and problematic Natural part of the course or curriculum

Four General Education Literacy areas: Language Literacy – Written Communication Analytic Literacy – Problem Solving Cultural Literacy – Composition of Art Social Literacy – Personal & Civic Responsibility

Advantages of Using Rubrics Clarity

Expectations, objectives, grading, feedback Objectivity

Standardized, consistent, fair, valid, reliable Legitimacy

Fairness increases student responsibility Efficiency

Easy to make, use and explain Improve skills & End Products

Instructor, students, peers

Writing Rubrics Identify and define the assessment

objective or purpose Select and write the needed number of

scoring criteria Select and write the desired levels of

performance If desired, select and write the

descriptors

Electronic Rubric Builders

Here are a few: Teachnology.com Rubistar Rubric Studio

Calibrating Rubrics: Validity Validity – the accuracy with which the

rubric assesses the objective or purpose; are we evaluating what we intended? Self Check Colleague Review Student Review Pilot Test

Calibrating Rubrics: Reliability Reliability – how consistently the rubric

assesses the objective or purpose over time and across raters; are the resulting scores consistent? Interrater – consistency of scores assigned by two

or more raters Intrarater – consistency of scores assigned by same

rater at different points in time External factors – time of day, order of assignments Internal factors – mood, fatigue, attention

End-of-training Quiz What have we learned?

Please complete the 10-item online quiz. It is a quick formative assessment of what we’ve covered today.

Upon completion, you will receive a feedback page with your final score. Please print a copy of this page and forward it to your Dean or VP as evidence that you have completed the assessment training.

Rubric Resources Documenting Excellence – General Education Rubrics:

http://www.documentingexcellence.com/examples/rubricgened/rubric.htm

Authentic Assessment Toolbox: http://jonathan.mueller.faculty.noctrl.edu/toolbox/rubrics.htm

Rubrics.com: http://www.rubrics.com/

Teacher Created Rubrics for Assessment: http://www.uwstout.edu/soe/profdev/rubrics.shtml

Sinclair Community College – General Education Rubrics: http://www.sinclair.edu/about/gened/genedrubrics/index.cfm

CSU – Institutional research Assessment & Planning: http://www.csufresno.edu/ir/assessment/rubric.shtml

References Moskal, B. M., & Leydens, J.A. (2000). Scoring rubric development:

Validity and reliability. Practical Assessment, Research & Evaluation, 7(10). Retrieved June 19, 2007 from http://PAREonline.net/getvn.asp?v=7&n=10

Practical Assessment, Research and Evaluation. A Peer-reviewed electronic journal. ISBN 1531-7714. http://pareonline.net

Richards, K. (2005). A primer on rubrics. Retrieved, June 15th, 2007 from http://online.lsc.mnscu.edu/Academics/assessment/Documents/Primer_on_Rubrics.pdf

Stevens, D.D., & Levi, A.J. (2004). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback and promote student learning. Stylus Publishing.