Download - Assessment FOR Learning: Rubric Development Transforming Our Teaching And Learning Module 2
Assessment FOR Learning: Rubric Development
Transforming Our Teaching And Learning
Module 2
Do you hear this at your school?
Why did I get a B for this assignment?
You didn’t tell us that spelling counts.
Will this be on the test?
Why do we have to learn this?
My friend said my “A” in America History isn’t the same as his “A”.
Purpose for Module 2
•Give teachers an understanding of formative assessment
•Present a framework and tools for creating and using rubrics for assessing student learning
Introduction Professional
Development Modules Hawaii Standards Implementation Process
Six Step Model Identify Relevant
Standards Determine Acceptable evidence & criteria
Determine Learning Experiences
Teach and Collect Evidence
Assess student work/inform instruction
Evaluate/judge/ communicate learning results
#1. Standards Implementation Process
HCPS Toolkit:
Curriculum Frameworks
Instructional Guide
:
The process
6 steps
Cyclical Nature
Built on system of standards
#2 Assessment FOR Learning: Rubric Development
HCPS Toolkit:
Curriculum Framework
Instructional Guide
Learning driven by
standards
Know when
learning has been achieved
Know what learning looks like
Evidence aligned to curriculum and instruction
Use information for goal setting
#3 Curriculum Mapping
HCPS Toolkit: Pacing Guides Instructional Guide
Standards Pacing Guides as a resource
Curriculum maps
and assessment data
School and
district reform approach
Collect data
regarding actual teaching
#4. Standards-Based Grading and Reporting
HCPS Toolkit RC FAQs/Website Parent Brochure
Assessment Grades
Based on Standards
Alignment with
standards based instruction and assessment
Grading
Practices Purpose for
Grading Special
Needs
Guidelines HI Implementation
Desired Outcomes
Teachers will review their understanding of quality assessment and its critical role in instruction.
Teachers will know how to identify and develop appropriate criteria for use in rubrics and checklists.
Teachers will know a process to develop performance assessment rubrics that are valid and reliable and inform instruction.
Identify relevant content standards
Determine acceptable evidence and criteria
Determine learning experiences that will enable students to learn what they need to
know and to do
Teach and collect evidence of student learning
Assess student work to inform instruction or use data to provide feedback
Evaluate student work and make judgment on learning results and
communicate findings
Reteach, or repeat the process with the next set of relevant standards
Teach
er c
olla
bora
tion
thro
ug
hou
t the
pro
cess.
Stu
den
t involv
em
en
t thro
ug
hou
t the
pro
cess.
Adapted from WestEd’s Learning from Assessment
Assessment FOR Learning Journey
= Rest Stop, time for reflection
Key 1: Purpose
Key 3: SoundDesign
Key 4: Communicat
ionRubrics
Key 2: Targets
Key 5: Student Involvement
Key Concepts
•The best performance assessment tasks are interesting, worthwhile activities that relate to instructional outcomes and allow students to demonstrate what they know and can do.
•Sound criteria should truly reflect the nature of quality in a product or performance.•Rubrics communicate in concrete and observable terms what the school values most. •Training students to develop and use rubrics to score their own work can be a powerful instructional tool.•The validity of generalizations we can make about student learning is increased when assessments are reliable, fair, and linked to instruction.
Agenda
•Assessment FOR Learning: 5 Keys
•Purpose: Seven Strategies
•Target-Method Match
•Deconstructing Standards
•Communication: Descriptive Feedback
•Student Involvement
•Reliability/Validity
ActivityActivityPart 1 Assessment OF
Learning or
Assessment FOR
Learning?Think of every type of
assessment used in your
classroom or at your
school. Write them on
post-its (one per post-it)
and set them aside.
With a partner:
Think of a negativenegative experience you’ve had being assessed. What made it negative?What made it negative? What was the effectthe effect on you?
Now, think of a positivepositive experience. What made it positive?What made it positive?What was the effectthe effect on you?
Negative Experiences
Causes Not clear what was to be tested Feedback method embarrassing Feedback too late to do any good Incomprehensible feedback/no
feedback Results didn’t reflect knowledge Trick questions No chance to improve/
one shot do or die Personal attack Time limits
Effects Stopped trying Never talked in class
again Changed majors Never took another
class in this subject Embarrassment Anger Redoubled efforts
Positive ExperiencesCauses Clear what was to be tested Criteria for success were clear Feedback could be used to improve
performance Personalized feedback Practice similar to assessment even
if not graded Step by step learning aligned with
assessment Questions were understandable Chance to improve
Effects Felt successful Wanted to take
more classes on this subject
Felt encouraged to keep trying
Knew what it took to succeed
Motivated to learn Redoubled efforts
Assessment Philosophy
We believe that assessment can do more than merely document student achievement. We believe that assessment, if approached properly, can nurture the very achievement that we are trying to document.
Judy Arter and Bill Savard
CLASSROOM ASSESSMENT
Formative Assessment
(for Learning)
Summative Assessment
(of Learning)
Purpose
Performance Assessment
Selected Response
Personal Communicatio
nEssay
Type
Purposes: Assessment OF/FOR Learning
DVD: A Hopeful Vision of the Future
Use handout #1 to take notes
ActivityActivityAssessment OF
Learning or
Assessment FOR
Learning?View the Rick Stiggins
video and complete
Figure 2.1. After
viewing the video, place
your assessment post-its
where they belong on the
FOR and OF chart in your
handout packet
Part 2
What we choose to evaluate and how we choose to evaluate delivers powerful
messages to students about those things we value. Students view their learning
and their sense of worth through the lens we help them construct unless they
cannot bear to look through it. “
Staytor and Johnson, 1990
When to Use Each
Discuss when to use Assessment OF Learning Assessment FOR Learning
7 Strategy Handout Jigsaw Read Discuss Implications
Assessment for Learning StrategiesAssessment for Learning StrategiesKey 1: PurposeKey 1: Purpose
Where am I going?Where am I going?
1. Provide a clear statement of the learning target
2. Use examples and modelsWhere am I now?Where am I now?
3. Offer regular descriptive feedback4. Teach students to self-assess and set goals
How can I close the gap?How can I close the gap?
5. Design focused lessons6. Teach students focused revision7. Engage students in self-reflection; let them
keep track of and share their learning
Key 2: Targets
Without clear targets we can’t:Without clear targets we can’t:
Know if the assessment covers what we taught.
Make a good inference about level of student achievement.
Plan the next steps in instruction. Keep track of student learning target-
by-target or standard-by standard. Give descriptive feedback to students. Have students self-assess. Complete a standards-based report
card.
Key 2: Clear TargetsWhat’s the Target?
Is This a Target? Senior project Bird feeder Use a band saw
safely State report Diorama
A Mathematics Example
Math Decimals Pages 152 in the
book Going on a “decimal
hunt” Read decimals and
put them in order
Kinds of Achievement Targets
Master Factual and Procedural Knowledge
Use Knowledge to Reason
Demonstrate Mastery of Skills
Create Products
Acquire Dispositions
Knowledge VerbsKnowledge Verbs
ExplainUnderstandDescribe IdentifyListDefineRecallRecognizeSelect
Kinds of Achievement Targets
Master factual and procedural knowledge
Understands long-term physiological benefits of regular participation in physical activity
Explains the important characteristics of U.S. citizenship
Knows that energy can be transformed between various forms
Reasoning VerbsReasoning Verbs
Predicts Infers Classifies Hypothesizes Compares Summarizes Analyzes Evaluates Generalizes
Kinds of Achievement Targets
Use knowledge to reason and solve problems
Uses statistical methods to describe, analyze, evaluate, and make decisions
Analyzes fitness assessments to set personal fitness goals, strategizes ways to reach goals, evaluates activities
Examines data/results and proposes meaningful interpretation
Skill VerbsSkill Verbs
ObservePerformDo ConductReadSpeakOperateInvestigateCollect
Kinds of Achievement Targets
Demonstrate mastery of specific skills Measures length in metric and US units Reads aloud with fluency and expression Dribbles to keep the ball away from an
opponent Participates in civic discussion with the
aim of solving current problems Uses simple equipment and tools to
gather data
Product VerbsProduct Verbs
DesignProduceMake WriteDrawRepresentDisplayModel
Kinds of Achievement Targets
Create quality products Constructs bar graphs Develops a personal health-related
fitness plan Builds physical models of familiar
objects Creates a scripted scene based on
improvised work
Kinds of Achievement Targets
Acquire positive affect Sees self as capable of doing mathematics Chooses to read for enjoyment Intends to vote in every election Looks forward to science class Wants to participate in community theater Enjoys opportunities to converse in
Spanish
GENERAL LEARNER OUTCOMES
1. Self-Directed Learner2. Community Contributor3. Complex Thinker4. Quality Producer5. Effective Communicator6. Effective and Ethical User of
Technology
Targets: Deconstructing
Standards
Your Turn…What Kind of Target Is This?
• Interpret text(s) from multiple perspectives… (6-8)• Create and use surveys, interviews, polls, … (K-3)• Design and conduct simple investigations… (4-5)• Explain how sanitary practices, vaccinations,
medicines, … keep people healthy (K-3)• Identify and use appropriate gestures and other
forms of non-verbal communication (4-5)• Sing expressively with appropriate dynamics,
breath control, phrasing, … (6-8)
What Kind of Target Is This?
• Relate critical facts and details in narrative or information text to comprehend text (2-3)
• Conveys meaning, provides important information, makes, a point, fulfills a purpose (writing, 4-5)
• Explain and apply tools and methods drawn from political science to examine political issues and/or problems (6-8)
• Communicate and defend scientific explanations and conclusions (9-12)
• Describe and exemplify how information and communication technologies affect research and work done in the field of science (6-8)
• Sing expressively with appropriate dynamics and phrasing (K-3)
What from Key 2 can you take back to your classroom and use?What from Key 2 can you take back to your classroom and use?
Making targets for a lesson clear Making sure the mix of target
types is as wanted Putting learning targets in
student-friendly language.
Key 3:Designing an Assessment
Design Features
Select a proper methodproper methodCreate quality items/tasksquality items/tasksGather enough evidenceenough evidence Minimize biasbias
Five Standards of Assessment Quality
Standard 1Clear Targets
Standard 2Clear and
Appropriate Users and UsesStandard 3
Appropriate Assessment Methods
Standard 4Sampling
Standard 5Potential Sources of Bias and Distortion
TUMS Bias
T = Clear TargetsU = Appropriate Users/UsesM = Method of AssessmentS = SamplesBias = Minimize Bias
Key 3: Sound DesignPossible Assessment Methods
Selected responseExtended Written response (essay)Performance assessmentsPersonal communication
Task
Any activity, project, prompt, problem or assignment that is given to the student to do.
The quality of the student response to the task is what is assessed using the criteria.
Task Vs. Assessment
Assess How?—Assess How?—SoundSound Design Design
Assessment methods matched to learning methods matched to learning targetstargets
Well-writtenWell-written tasks, items, and scoring guides Adequate sampleAdequate sample Bias avoidedBias avoided
Selected Response Example
Benchmark: Explain the phases of the moon and eclipses.
A lunar eclipse occurs when:A. The moon passes between the earth and the
sun.B. The earth passes between the moon and the
sun.C. The sun passes between the earth and the
moon.D. Every month when there is a “new moon”.E. Both answers A and D.
Short Responses
Benchmark: Describe and explain the effects of multiple forces acting on an object.Prompt: There are at least two forces that affect the speed of your car, forward motion and resistance to forward motion.
Explain how these forces affect the performance of your solar car and give an example to support your answer.
Criteria for SelectingPerformance Tasks
Measure several standards or outcomes
Involve critical thinking skills Use instructional time wisely Focus on credible real world tasks Minimize bias and fair Feasible Clearly defined-- Adapted from Herman, Aschbacher and Winters (1992)
Personal CommunicationType of Target AchievementDispositions
Assessment MethodPersonal CommunicationCan talk with students about their feelings.
Fine Arts:Integrated Stds, Grade 1 Performance Indicator: Discusses personal feelings about a variety of works.
Make an animal sculpture with clay starting with a pinch pot. Tell how you feel about the creature that you made.
Key 3: Sound DesignKey 3: Sound DesignAdequate Sampling—the
Basics Can’t cover everything; need to
make inferences about student learning
Example: “Land a plane.”Example: “Write well for different
audiences and purposes.”
Sampling—What to doSampling—What to do
Gather enough evidence to draw confident conclusions—without—wasting time gathering too much
Ask: Does number of items/tasks for each target represent importance in instruction?
Ask: Do the items/tasks cover the target well?
Sound Design: Sources of Bias
AssessmentAssessorStudentEnvironment
Some generic Some method
specific
See p. 115-116 in CASL
Sound DesignSound DesignBias and Distortion Activity
Read through the potential sources of bias and distortion on CASL, pp 115-116.
Mark with a check (or list) any you have experienced—as the assessment giver or taker.
Two categories—those we control and those we don’t
Find a source of bias you checked that is one we can control
Talk with a partner about how you would go about eliminating or minimizing this source of bias and distortion—use
Analyze an Assessment for Quality: Keys 1, 2, and 3
Choose any one of the four sample assessments. Use the separate rating/checklist form to analyze it for quality.
Assessment Title Key 1: Purpose Key 2: Targets Key 3: Design
Mathematics
Fish Tank
Culminating Project
Emerson
Key 4:
Communication: Descriptive
Feedback
Key 4: CommunicationDescriptive Feedback
Describes features of work or performance
Relates directly to learning targets and/or standards of quality
Points out strengths and gives specific information about how to improve
CommunicationCommunication
Explains the strategy and processes used
Explains why what was done was done
Explains why answer works
FeedbackFeedback
Research backed Brown, 1994 Sadler, 1989 Black & Wiliam, 1998
Descriptive or Evaluative? You decide
Key 5:
Student Involveme
nt
Policy on Reporting Student Progress (4510)
Policy on Reporting Student Progress (4510)
“The involvement of the student in the evaluation process should be considered essential …”
“The progress report shall … be diagnostic and constructive …”
“Secondary students are encouraged to conduct regular parent-teacher-student conferences.”
Key 5:Student-Involved Assessment
Students … Are partners in development Describe expectations for success See how close they are now Create a clear path to success
What are the implications of involving students in assessment?
REST STOP
Key 5: Student Involvement
1. Identify an important learning goal, or one students have difficulty learning.
2. Identify word(s) needing clarification.3. Define the word(s). We use a dictionary as a
starting point.4. Rewrite the definition as an “I can” (or an “I
am learning to”) statement, in terms that your students will understand.
5. Try it out on students or a colleague and refine as needed.
6. Have students try this process for subsequent learning goals.
Converting Learning Targets to student-Friendly Language
The Process:
Standards-based Learning Goals
Summarize text, e.g.: Summarize important information in texts to demonstrate comprehension (4)
Make predictions, e.g.: Predict or hypothesize…from information in the text, … (6)
Summarize textSummarize textSummarize textSummarize text
Words to be definedWords to be defined: SUMMARIZE SUMMARIZE: to give a brief statement of
the main points, main events, or important ideas
Student-friendly language: I can summarize text. This means I can make a short statement
of the main points or the big ideas of what I read.
Make PredictionsMake Predictions
Words to be defined: prediction PREDICTION: A statement that something
will happen in the future. Student-friendly language:
I can make predictions. This means I can use information from
what I read to guess at what will happen next. (Or, to guess what the author will tell me next.)
Your Turn…Your Turn… Words to be defined: Infer
Definition: Student-friendly language: I can infer. This means I can…
Words to be defined: Hypothesize Definition: Student-friendly language: I can hypothesize. This means I can…
Make inferencesMake inferences
Word to be defined: INFERENCE INFERENCE: A conclusion drawn
from the information available Student-friendly language:
I can make inferences. This means I can use information
from what I read to draw a conclusion. (Or, I can use clues to make good guesses.)
Key 5: Student Involvement4-Step Process for Setting
Criteria with Students
Step 1: Brainstorm
Step 2: Sort & Categorize
Step 3: Create a T-chart
Step 4: Add, Revise, & Refine
Step 1: Brainstorm
Pose a question
Record all ideas on the chart
paper
Add in your own ideas to make
sure that all the important
features are included and your
goals are met
Step 2: Sort & Categorize
Have students look at the list and see if any ideas fit together. Ask “Do you see any similar items?”
Circle or mark the similar items using colored pens.
Decide on headings using big ideas.
Step 3: Create a T-Chart
Draw a T-Chart Transfer the items
from Step 2. Put the big ideas or
criteria on the left. Put the details or
specifics on the right.
Criteria Details
Step 4: Add, Revise, Refine
Have students review the T-Chart. Ask students, “ Are there any new
items or criteria that we need to add?”
Make changes as necessary.
Classroom Assessment Quiz ___Rubric
___Scoring Guide
___Criteria
A. A newly formed republic in eastern Europe
B. Your Aunt Frieda’s favorite cabbage recipe
C. Written statements that describe levels of performance
D. John Grisham’s latest novel
E. A 1950’s dating manual
Now that you have the criteria, what do you do with it??
What is a Rubric?
An assessment that evaluates student performance
Uses specific criteria for assessment
Defines levels of performance for each criterion
Performance Criteria are guidelines, rules or principles by which student responses, products or performances are judged. They describe what to look for in student performances or products to judge quality. They are useful in assessing criteria which are complex and subjective.
Rubric Development
Performance Criteria
MeetsProficiencyW/Excellence
MeetsProficiency
Approaches
ProficiencyBelowProficiency
PRODUCINGQUALITYWORK
Leaps tall buildings in a single bound
Must take a running start to leap over buildings
Can leap over short buildings
Crashes into buildings or cannot recognize them
JOBKNOWLEDGE
Always walks on water
Walks on water in emergency
Washes with water
Drinks water
COMMUNICATION
Talks with God
Talks with employees
Talks to him/herself
Loses arguments with self
Why Use Rubrics
Assessment is more objective and consistent.
The target is clearer – shows what is expected and how work will be evaluated.
It gives feedback on the effectiveness of the instruction.
It provides benchmarks against which to measure and document progress.
Common Features of Rubrics
Measures a stated objective (performance, behavior, or quality)
Uses a range to assess performance
Contains specific performance characteristics arranged in levels indicating the degree to which a criterion has been met.
What Student Involvement with Rubrics Doesn’t Look Like
Handing the rubric to students with little explanation.
No practice with the rubric before the “grading” use.
Asking students to self- or peer-assess using the rubric without teaching them how.
Using any old rubric without making sure it adheres to standards of quality.
Key 5:Student-Involved
Communication
Student-led parent/teacher conferences Greater sense of responsibility Pride in accomplishment Greater achievement
“Grades … must meet the dual criteria of validity and reliability. The test of validity is met when the grades have been based on HCPS. The test of reliability is met when (1) there is a sufficient number of grade entries … (2) records are maintained accurately.”
Policy on Grading (4501)
Validity
A good rubric must possess “validity.”Definition: the extent to which what is being
measured by an instrument is actually what is intended. Are the test and rubric actually measuring the desired performance-outcomes?
Reliability
A good rubric must possess “reliability.”
Definition: the extent to which the measuring
instrument yields responses that are consistent across time and between different scorers.
Reliability
Factors that have nothing to do with the purpose of the assessment can impact scores a student can get. Examples: Rater fatigue Rater’s mood Failing students may get more critical look than
“good” students Well-designed rubrics respond to this concern
by establishing a description of the scoring criteria in advance. Scorer should revisit the established criteria to ensure consistency.
Reliability (cont.)
Clarity of the rubric Are the scoring categories well-defined? Are the differences between scoring
categories clear? Would two independent raters arrive at the
same score for a given response? Use anchor papers. Whenever possible, share scoring rubrics
with students in advance.
Why Accuracy Is Important
Assessment information can be inaccurate in one of two ways:
We think students have learned material when they actually have not
We think students haven’t learned material when they actually have
What problems for What problems for teachers and teachers and
students would arise students would arise from each of these from each of these
situations?situations?
“We must change from a model that picks winners to one that will create winners.”Harold Hodgkinson, 1987“Michigan: The State and Its Educational System”
Observations of students in the process of learning
Communications- student interviews, journals, discussions, comments
Products- student work, tests, reports, models, performances
Triangulation of Evidence: Assessment Data from Many Sources
ActivityActivityTriangulati
onTakingTaking your post-its
from the earlier
Assessment FOR and
OF Learning chart,
make a triangulation
poster at your table.
Place the post-its
where they belong on
the poster.
What do you notice?
Reflecting on your practice: When developing assessments, what concerns or issues regarding triangulation of data need to be considered?
REST STOP
Decisions About Performance Criteria
Performance Criteria
Task Specific Generalized
Holistic (on overall Score)
Analytical Trait
Trait
(several dimensions scored separately)
Quantity or Quality
Amount of Detail
Number of Score Points
General or Task Specific?
GENERALIZED or Generic - can be used across similar performances.
TASK-SPECIFIC- each one can only be used for
a single task.
Holistic or Analytical?
HOLISTICgives a single score or rating for an entire product or performance based on an overall impression of a student’s work.
ANALYTICAL TRAITdivides a product or
performance into essential traits or dimensions so that they can be judged separately.
Holistic or Analytical Trait
HolisticUse : Quick snapshot of overall
status When speed of scoring is
important Simple products or
performances
Disadvantages: 2 students can get same
score for different reasons
Can’t identify strengths & weaknesses
Not useful for students
AnalyticalUse Planning instruction - show
relative strengths & weaknesses
Details to teach student quality Detailed feedback Precision more important that
speed:Disadvantages: Scoring is slower Takes longer to learn
Options for Selecting Rubrics
Create your own - build from scratch Adopt - use an existing rubric Adapt - Modify or combine existing
rubrics Reword parts Drop or change one more
scales Omit criteria not relevant “Mix” and Match” Divide a holistic rubric into
several scales Add a “no response” category Modify for different grade levels
Online Rubric Resources
Kathy Schrock at Discovery School has compiled an extensive list of online rubric resources and rubric builders at:http://school.discovery.com/schrockguide/
assess.html#builders
1. Individually read and score three student work samples (Practice Papers #01, #02, and #03) using the Sample Student Rubric (Handout 6).
2. Share the scores in your group.
3. Discuss the criteria on the rubric to develop common understandings. What do you notice?
REVISEREVISE-- Using the Scoring Criteria on the Item and Scoring Criteria sheet, revise the rubric to include criteria measuring Core Concepts as well as the criteria for Accessing Information.
READ and SCORE-READ and SCORE- Then each member of the group will read and score 2-3 of the remaining samples (Practice Papers #04-#08) using the revised rubric.
DISCUSS and REFINEDISCUSS and REFINE - Discuss the following questions in your group: If we were going to use this rubric again, would we want to add new criteria? Clarify existing criteria? Delete insignificant criteria?