How to Create Meaningful Rubrics for Student Assessment

Download How to Create Meaningful Rubrics for Student Assessment

Post on 08-Feb-2017




3 download


Slide 1Creating Meaningful Rubrics for Student AssessmentAimee F. Strang, Pharm.D., MSHPEdAssistant Dean of Curricular AssessmentObjectivesRecognize common flawsDescribe characteristics of good criteriaWrite descriptions that distinguish levels of performanceUse ExamSoft to assign gradesEnsure rubrics evaluate learning2By the end of this presentation, I hope you wil be able to 1/25/1723CriteriaLevels of PerformanceBenefits of RubricsPerformance assessmentAuthentic assessmentScoring guideGrader consistency Guide student learningFormative assessment4Im assuming you have experience using them. If not, you are interested in them because your listening today. Either way you know a bit about them.Rubrics is a coherent set of pre-defined criteria that include descriptions of levels of performance quality. Help teacher teach - Help coordinate instructionHelp students learn - understand what is expected of them and at which level they are performingHelp students self-improve by comparing their work against the rubricImprove reliability of rating or scoring student learning by keeping the rater focused on what is importantGreat tool IF used ot written correctly. Lots of BAD rubrics out there1/25/174Part I: How to Create a Rubric5From scratch part 2 is NOT to create .. .good for revision of existing rubrics1/25/1751. Focus on the Learning, Not the ActivityStart with the learning objectivesCreate an assignment that demonstrates performance of understandingBe wary of engaging busy workLO: Analyze current trends that affect faculty and institutions in academic health professionsCreate a jeopardy gameCreate a poster presentation6Pre-rubric step My course- Trends in Academic Health. Objectives - Analyze current trends in academic healthBoth game and poster- just list the trends, present facts and figures both are engaging but neither meet the LO analyzeAnalyze - Organizes and synthesizes evidence to reveal insightful patterns, differences, or similarities related to focus. 1/25/1762. Select Good Performance CriteriaDemonstrate evidence of learning or meeting objective (not achievement of assignment tasks)Criteria can be named, defined, described and observedVary along a continuum of qualityUnidimensional or distinctLO: Triage a patient to the appropriate level of careHistory Medication HistoryTriageCommunication7Not yes or No checklistMedication choice and counseling not unidimentional 1/25/1773. Differentiate Levels of Performance Start with a description of minimum proficiencyDescribe a continuumYes Yes but No but NoHow many levels?LO: Recommend most appropriate cold medication to a patientMedication choice8Describe performance what would you observe? NOT how would you evaluate the work (Descriptive NOT judgmental)Dont use Excellent, good fair, poor1-4 middle is catchallChoose a medication that addressed all symptoms and met patient needsChoose med that addressed all sx but was not the best choice (cost, doing etc)Choose a med that only addressed some sxChoose wrong medication1/25/1784. Quality CheckGrade past assignment Get peer feedbackGet student feedbackMake improvements each time9Part II: How NOT to Create a Rubric10 Dont Measure More Than 1 Criteria at a Time Originality Develops a project that is unique specifically targeting a select population Project presentation delivery is designed for a select population Develops a project that targets a select population Project presentation delivery is generic Develops a generic project Project does not target a select population Professional Language Appropriate use of professional language and lay terms identified as necessary for the targeted population Spelling and grammar are correct The balance between professional language and lay terms is inappropriate Minimal spelling and grammar mistakes Unprofessional language is used often (texting language) Many spelling and grammar mistakes 11Confounding outcomes- measuring more than one outcome without recognizing them as separate skillsAssignment was to create health and wellness education for a target population. Originality AND AudienceProfessional language AND GrammerGo back to learning objectives what is the learning goal? : Provide prevention, intervention and educational strategies for select populations to improve health and wellness-Assess the healthcare status and needs of a targeted population. Communicate assertively, persuasively, confidently and clearly. 1/25/1711 Dont Create Criterion for Nonessential FeaturesNeatnessDesign of cover pageAudience engagement Format of references121/25/1712 Dont Score by Counting13Critical Elements Missing four or more of the following: Background & description Market analysis and strategy Operational structure and process Financial projections and feasibility Action plan for implementation Critical risks and opportunities Conclusion/summaryMissing less than 4Includes all of the followingRubric used to gradePharmacy administration course assignment was to create a business plan for a new pharmacy(checklist) no levels of criteria1/25/1713 Dont Score Things Not Part of the RubricProfessionalism Acted in a professional manner (eye contact, empathy, open posture)Acted professional but needs some improvementActed professionally but need significant improvementDid not act professionally14EffortNervousnessConfidence Language1/25/1714 Dont Score the Product15AssignmentRubric CriterionReflect on todays learningStages of reflectionWrite paper on history of pharmacyOrganization, spelling, lengthCounsel patient on use of eye dropsIntroduction, body language, eye contact Dont Codify Assignment Directions into RubricPerformance Level Needs Improvement Satisfactory Excellent Contribution and Participation Not apparent that all the students in the group contributed. One or two people appear to have done all the work. Student dissent on levels of contribution. Apparent that some of the students contributed a fair share to the project, though some workloads varied. Students made an effort to include all group members in the process. Apparent that all students contributed equally to the project. Evident that students worked with each other in a cooperative manner and split the project to match the students strengths. 16Grade is an evaluation of compliance with the rubric or assignment directions, not of learningWork in groups to create a SOAP note using evidence based medicine1/25/1716 Dont Evaluative17Main function is to match performance to a description, not judge it immediately. The description is the BRIDGE between student work and evaluation or judgement of learning. 1/25/1717 Quality CheckGet 10-20 copies of student past workSort in to 3 categories of quality (Excellent Good Poor) Write descriptions for why each piece of work is categorized Compare and contrast work to identify relevant criteriaWrite descriptions of quality along a continuum for each criteria 181/25/1718Part III: Converting Rubrics to Grades19Assigning Grading SchemasAlign levels with grading (A, B, C, D, F)Proficiency levelsExceeds, Meets, Needs Improvement, Significant DeficiencyBeginner, Developing, Proficient, AdvancedMinimum levels of acceptability = 60%Weighting criteriaTopic selection, Background, Analysis, ConclusionPoints vs Range20How many levels should there be use as many id necessary to describe meaningful differences in performance quality1/25/1720Using Points21FailingProficientAdvancedMinimum proficiency = Passing = 60%1/25/172122NoviceExpertAdvancingRubric ARubric BWeighting Criteria23Rubric BRubric AUsing Ranges24Determine RangesEnter GradeConclusionRecognize common flawsDescribe characteristics of good criteriaWrite descriptions that distinguish levels of performanceUse ExamSoft to assign gradesEnsure rubrics evaluate learning25By the end of this presentation, I hope you wil be able to 1/25/1725Questions?Contact Information:Aimee Strang, Pharm.D., MSHPEdAssistant Dean of Curricular AssessmentAlbany College of Pharmacy and Health Sciences518-694-7320 | 26