how to create rubrics - university of north texas at dallas · student’s oral presentation...
TRANSCRIPT
HOW TO CREATE ANALYTIC RUBRICSELIZABETH O. GIDDENS, PH.D.
WHAT IS A RUBRIC?
A rubric is a scoring tool that lays out the specific expectations of an assignment or goal. Rubrics break up the
assignment/goal into smaller parts, and provide a detailed description of what constitutes acceptable and
unacceptable levels of performance for each part (Stevens and Levi, 2013).
WHAT DOES A RUBRIC LOOK LIKE?
An analytic rubric is a table/matrix of criteria and
their descriptors (UC Denver and UConn):
The left side of a rubric matrix lists
the dimensions. The dimensions are the
components/parts of an assignment or project.
Across the top of the rubric matrix is the rating
scale. The values could be “above” “meets” and
“below.” They also could be, and often are,
numerical (3, 2, 1).
Descriptors under the rating scale and to the
right of the dimensions provide written
descriptions for each level of performance.
1 Below 2 Meets 3 Above
Grammar The author makes
numerous errors in
grammar, mechanics,
and/or spelling, and it
interferes with the
reader’s understanding.
The author makes
some errors in
grammar, mechanics,
and/or spelling, but it
does not interfere with
the reader’s
understanding.
The author makes no
errors in grammar,
mechanics, or spelling.
Example of a dimension of a rubric about a paper
REMINDER OF WHO DOES WHAT TYPE OF ASSESSMENT…
Educational Assessment Student-Service Assessment Administrative Assessment
• Faculty • Experiential Learning
• Distance Learning
• Library
• Learning Commons
• Academic Advising
• Admissions and Orientation
• Financial Aid
• Leadership
Development/Multicultural
• Disability Services Office
• Career Services
• Veteran’s Success Center
• Counseling and Wellness
• Residence Life
• Police and Parking
• Development and Foundation
• Sponsored Projects
• Marketing and Communication
• President, Provost, and Deans
• Registrar
• Facilities
• Budget and Planning
• Financial Services/Accounting
• Strategic Analysis/Reporting
• Human Resources
• Risk Management
• Information Technology
• Athletics
WHY SHOULD YOU USE A RUBRIC?
As an instructor, rubrics can help you communicate your expectations to students and assess student work
consistency and efficiently. Rubrics can also provide students with informative feedback on their strengths and
weaknesses, and prompt students to reflect on their own work. They can also help you design your assignments
(because they give you a chance to really think about what you want from the students).
As a student-service staff member, rubrics can help you determine the effectiveness of your program and services.
They can also help you measure your alignment to CAS Standards (more on this later).
As a administrative staff member, rubrics can help you judge the quality of a product or project. They can also help
you prioritize your needs.
FOR EXAMPLE (FACULTY)
This rubric was created to assess a
student’s research paper (Carnegie
Mellon).
ANOTHER EXAMPLE (FACULTY)
This rubric was created to assess a
student’s oral presentation
(Northern College).
FOR EXAMPLE (STUDENT-SERVICE STAFF)
This rubric was created to assess a
student’s intercultural awareness,
perhaps after intensive programming
(UNC).
ANOTHER EXAMPLE (STUDENT-SERVICE STAFF)
This rubric was created to assess a
student’s career-planning skills and
study-success skills, perhaps after a
1:1 meeting (Paul Gordon Brown).
FOR EXAMPLE (ADMINISTRATIVE STAFF)
This rubric was created to
determine which software they
wanted to purchase and
implement (Western University).
ANOTHER EXAMPLE (ADMINISTRATIVE STAFF)
This rubric was created to
determine priorities for
deferred maintenance (MTU).
ANOTHER EXAMPLE (STUDENT-SERVICE STAFF) – CAS STANDARDS
This rubric was created to see if a
department aligned with the CAS
Standard about “Ethics” (CAS).
HOW TO MAKE A RUBRIC
1. Decide on a manageable number of important dimensions of the assignment/goal.
2. Decide how many performance levels you would like to include. Decide on the labels for each performance level. Examples:
Excellent, competent, needs work (3 levels)
Accomplished, proficient, developing, beginning (4 levels)
Excellent, good, adequate, needs work, incomplete (5 levels)
3. For each dimension, describe the highest performance. Be as specific as possible.
4. Define the remaining performance levels (“good”, “adequate”, “needs work”, etc.) for each dimension. Be as specific as possible. Describe each level as completely as you can. For example, make sure to specify how “good” is different from “adequate” or “excellent” (Laney).
5. Optional Step:
A. Using rubrics for grades. See video here starting at 4:03: https://www.youtube.com/watch?v=jSwbR1oDXLY
B. Adding weights to each dimension so some are scored more heavily than others toward the final grade. See article here: https://www.teachervision.com/how-weight-rubrics-part-four-five-part-series
LET’S DESIGN ONE FOR “MAKING BROWNIES”
I made brownies! To do this, I…
1. Washed hands to ensure hygiene
2. Gathered and measured ingredients
3. Followed recipe for collating ingredients,
including stirring and baking
4. Cut, presented, and served completed dish
5. Tasted the final product
See handout. These five steps became my dimensions; you
can choose different ones if you want. Also, I already
included a 1-3 rating scale. In groups, fill out the descriptors.
TIPS FOR WRITING DESCRIPTIONS…
No rubric is perfect, but let’s try to make it as
perfect as we can. Here’s how:
Don’t write too much. Don’t write too little.
Ensure each level is different from the other so a
student cannot overlap into two scores.
Avoid vague terms.
Be consistent. Use parallel sentence construction to
describe the same content across different
performance levels for each dimension. For example,
see to the right (Megan Oak Leaf).
USING PARALLEL SENTENCE STRUCTURE
PARALLEL EXAMPLE
NOT PARALLEL EXAMPLE
IN GROUPS, WORK ON YOUR RUBRIC FOR 10-15 MINUTES
IS THE RUBRIC YOU MADE ANY GOOD?
To the right is a quality control rubric that
can be used to judge the quality of the
rubric you made (Stevens and Levi, 2013)
– handout provided.
NO RUBRIC IS PERFECT, BUT THIS IS HOW WE CAN WORK
TOWARDS IT…
Reliability
Reliability is the extent to which an instrument (your rubric) yields consistent results when used repeatedly under the same conditions.
When two different graders use the reliable rubric on the same performance, they'll give similar scores. This is called inter-rater reliability.
Your rubric should also yield consistent results if one grader uses the same rubric to judge the same performance at different times. This is called intra-rater reliability. (UC Denver)
Validity
Content Validity refers to whether a content expert would agree with the criteria/dimensions chosen in the rubric. For example, if your rubric is assessing a student's ability to write a thematic essay, are your dimensions composed of the essential elements of a thematic essay?
Criterion Validity relates to the predictability of student performance. If a student gets a high score using your rubric as the measurement tool, can you also infer that the student will perform that task well on another similar test?
Face Validity refers to whether the rubric appears to be valid to its users. Students/others have a stake in the quality of your rubric, so it's important that the rubric seems right to them. (UC Denver)
SOURCES
Steven and Levi, 2013, “Introduction to Rubrics,” 2nd Edition
https://studentaffairs.unc.edu/faculty-staff/student-affairs-assessment/resources/survey-items-and-rubrics
http://www.northernc.on.ca/leid/docs/ja_createrubric.pdf
https://laney.edu/assessment/how-to-create-rubrics-and-examples/
http://www.ucdenver.edu/faculty_staff/faculty/center-for-faculty-development/Documents/Tutorials/Rubrics/1_what_is/index.htm
https://assessment.uconn.edu/wp-content/uploads/sites/1804/2016/06/How_to_Create_Rubrics.pdf
https://www.cmu.edu/teaching/assessment/assesslearning/rubrics.html
https://www.mtu.edu/facilities/pdfs/2011-mtu-facility-assessment-report-2011-10-13-update.pdf
https://www.cas.edu/Files/Presentations/CAS_Basics_082015.pdf
http://meganoakleaf.info/writingrubricsright.pdf
https://teaching.uwo.ca/pdf/elearning/Rubric-for-eLearning-Tool-Evaluation.pdf
https://www.youtube.com/watch?v=jSwbR1oDXLY
https://www.teachervision.com/how-weight-rubrics-part-four-five-part-series
https://www.thoughtco.com/essay-rubric-2081367
THANK YOU! QUESTIONS?