coherence cas cop assess ii
TRANSCRIPT
Finding CoherenceFocus, Measure, & Connect
The Measure Sessions!January 20th, 2015http://digitallearningforallnow.com
http://www.slideshare.net/jpcostasr
Jonathan P. Costa
Three Principles of Coherence
Measure what you
value
Value what you measure
Priority Student Learning
Priority Adult
LearningPriority Systems Learning
Student Learning
Adult Learning
Systems Learning
MeasureFocus Connections
Coherence Pathways
G = Goals P = Practices M= Measures Jonathan P. Costa S= Students A = Adults O = District/Building
Mission
To develop in all children the
knowledge, skills, attitudes and
values...
Theory of Action
FocusMeasureConnect
Student DataDriving
InstructionalPractices &
DecisionMaking
InstructionDriving
ImprovedMeasures
ProfessionalPracticeDriving
ImprovedMeasures
SystemsDriving
ImprovedMeasures
Adult DataDriving
ProfessionalLearning &
DecisionMaking
OrganizationalData
DrivingSystemsDecisionMaking
Coherence Is Supported by Assessment
Jonathan P. Costa
Value
Reliability
Align Assessments With Goals for Learning
Appropriate
Assessment
Instruments
Recall & recognition
based quizzes, tests,
and activities. Multiple
choice, matching, fill
In the blanks, etc.
(SAT/AP/Exams)
Checklists,
analytic rubrics,
or other agreed upon
skill standards
(AP/Smarter B/Exams)
Holistic and,
analytic rubrics,
or other agreed upon
skill standards
(Portfolios, Demonstrations,
Exhibitions, Etc)
Content(Declarative)
Facts
“Define this term”
Content Skills(Procedural)
Discrete Skills
“Factor this equation”
Meta Skills(Contextual)
Applied Understandings
“Skilled Information Consumer”
Type of
Knowledge
Desired
Appropriate
Assessment
Standards/Scales
Take this quiz, test,
or other appropriate
content focused
assessment instrument.
Answering/responding to
classroom or text based
problems, experiments, and
writing assignments.
Complex projects,
real time explorations,
authentic problem based
skill applications.
Appropriate
Assessment
Tasks
Percentages of
right and wrong
answers on an agreed
upon numeric or letter
grade scale.
1-100/A-B-C-D
Specific criteria tied
to an agreed upon
numeric or
descriptive scale.
1-100/Pass-Fail
Insufficient-Excellent
Specific criteria tied
to an agreed upon
descriptive scale.
Insufficient-Sufficient
Proficient-Excellent
Overlap between columns can and does occur.
Reflections on Assessment…
Other data points: Required, by grade, department, standards aligned, type of administration, time required, testing window, vendor, ROI
Reflections on Assessment…
To be Effective, Formative or Summative Assessments Should Be:
1. Generally Reliable (as opposed to statistically so)
2. Valid (reflects what it is intended to measure)
3. Helps the learner improve his/her performance (formative mostly)
4. Can be tracked and analyzed over time (summative mostly)
Building Technical Capacity
• Constructing effective rubrics – measuring what you value
• Scoring protocols – build capacity to judge student work
As a General Rule:
• The goal is the reduce the scoring range – to increase the reliability of the feedback
• Transparency with the standards – sharing standards with the learners and providing them with effective feedback to improve performance.
The ParetoLeverage Principle
VitalFew
Some things are more
important than others.
20/80
Pareto Is Not a New Idea
I served with General Washington in the Virginia Legislature and with Dr. Franklin in Congress. I never heard either of them speak to any but the main point that was to decide the question. They laid their shoulders to the great points, knowing that the little ones would follow themselves.
Thomas Jefferson
The Biggest Challenge
Getting together and building the structures from which to give students the feedback they need to ensure success and accountability in these priority skill areas is the hardest part of this work.
Please write 5 words that describe this winter’s weather in Connecticut.
Keep this shift in mind…
- From -View Judge Explain
- To -Establish a reliable standard; then…
View Analyze Discuss
It’s our natural instinct…Think back to the five words I asked you to
write to DESCRIBE the winter weather in Connecticut…
Pay close attention to the language.
Judgment - Descriptive“reliable” vs. “corroborated by at least
one other source”
“effective” “1st person narrative”
To help this shift…
Narrowing the Range
• As a general rule, descriptive language is preferable to judgmental language for the purpose of narrowing the range of scores.
• The more judgmental the language, the greater the time investment required for calibration and the more dependent on personnel the process.
• The goal is find the balance time – descriptive enough to effectively communicate without being too restrictive or unrealistic.
Getting Some Descriptive PracticeWhat is it – why is it seen that way?
Element
Spring day…
Turkey sandwich…
Student engagement…
Written summary…
Argument with evidence…
More DescriptiveMore Judgmental
Making an Argument Based on Evidence
Item Insufficient Sufficient Proficient Excellent
Stated Position
Rationale
Supporting
Evidence
Closing
Statement
Something to Think About• There is no shortcut to the amount of time
required to use standards and assessment effectively.
• You either have to define the standard from scratch through descriptive language or you will have to define it through working with your peers to develop anchor sets/models of student work.
• The best strategy is probably a combination of both approaches.
Scoring Practice
• Match evidence in the student work to the descriptors in the rubric and scoring guide.
• Scoring decisions must be based on particular evidence in the student work. Be able to point to the evidence in the student work that is relevant to scoring fairly and consistently.
• You must be indifferent to the student – about what is demonstrated not about what you believe was intended.