1

Self-assessment, Peer Review & Feedback

Testing Design Work Against Standards

2

Self-Assessment & Self-Adjustment

3

The research jibes with this common sense:

To achieve any goal you must learn how to self-assess your learning and your performance

One of 3 chief findings in How People Learn

Success depends upon valid self-assessment and timely self-adjustment on your own you with minimal outside prompting and close supervision

4

From the research: one of 3 chief findings -

“The teaching of metacognitive skills should be integrated into the curriculum. Because metacognition often takes the form of an internal dialogue, many learners may be unaware of its importance unless the processes are explicitly emphasized by teachers.

“Research has demonstrated that learners can be taught these strategies, including the ability to predict outcomes, explain to oneself, note failures to understand, activate background knowledge, plan ahead, and apportion time and memory…

How People Learn, p. 14, 21

5

Research on metacognition, part 2

The model for using the meta-cognitive strategies is provided initially by the teacher, and learners practice and discuss the strategies as they learn to use them.

Ultimately, learners are able to prompt themselves and monitor their own comprehension without teacher support. [transfer]

How People Learn, pp. 18-19

6

So, what follows for staff?

Models, feedback, adjustments, feedback

This cycle has to be woven into professional development and team meetings

This is the essence of LessonStudy, the highly successful Japanese approach

Depends on model units, ubd standards, a self-assessment protocol, and practice in self-assessment after having it modeled

Unit Design Cycles Unit Design CyclesBackward Design: • Stage 1• Stage 2• Stage 3

based on: • Program goals• Performance gaps

Feedback

Feedback

7

Analysis of

student work

Expert review

Student

feedback

Designer’s observation

Reviewed against Design Standards by: • self • peers

Design

Design

Trial

Trial

Working smarter via:• design teams

8

9

See the design standards and review materials in the UbD Workbook:

design standards linked to the three stages ‘backward design’

each standard contains criteria, a three-point performance scale and

indicators serve as guides for peer & external reviews

UbD – Design Standards

10

Peer Review: Goals Provide specific feedback and guidance to designer based on design standards.

Improve the quality of unit designs.

Increase understanding of the qualities of effective curricular design.

Develop a more collaborative and open culture

11

Peer Review Process

Step 1 - Overview of unit

Designer presents a brief overview of unit and requests specific feedback from the group. Designer leaves the group.

Suggestion: Designer should take another unit to review while they are waiting.

12

Peer Review Process

Step 2 - Individual reviews of unit

designs without designer present

Establish group roles.

Reviewers individually (and silently) review and evaluate the design based on review criteria and record comments about strengths and weaknesses on the Individual Review Form.

13

Peer Review Process

Group roles:

Timekeeper/Facilitator Recorder Spokesperson(s)

14

Peer Review Process

Step 2 - Individual reviews of unit

designs without designer present

Establish group roles.

Reviewers individually (and silently) review and evaluate the design based on review criteria and record comments about strengths and weaknesses on the Individual Review Form.

15

Peer Review Process

Step 3/4 - Review group discussion

without designer present

Review group discusses the design and records key questions, feedback and guidance on the Group Review Form.

Spokesperson(s) verbally summarizes the group’s comments.

16

Peer Review Process

Step 5 - Group discussion of unit with designer

Reviewers present feedback and guidance to the designer.

Designer listens, takes notes, and asks clarifying questions.

17

Feedback

Feedback is descriptive. It provides commentary on the extent to which the design meets the stated goals (intent vs. effect).

e.g., The first performance task does not seem to align with any of the unit Understandings.

18

Feedback, defined

Useful information on the actual against the optimal;

the intent vs. effect Feedback reports back on what you did, against a specific target—no personal value or aesthetic judgment is made Feedback is descriptive, not evaluative Feedback is not praise or blame Feedback is not guidance/advice

19

Examples of feedback about ubd

“That question seems to me to be the most ‘essential’ question - it is thought-provoking and can profitably be asked and re-asked throughout the unit .”

“That second one is really more of a leading factual question than an essential question.”

“I cannot find any assessment of your 2nd understanding”

“Those activities align nicely with Understandings 1 and 2. But I don’t see any activities directly related to #3.”

“I think a student could do well on that performance assessment without really understanding understanding #2 (which you say it addresses).”

20

understandable to the receiver timely

The most useful feedback is:

specific guided by criteria

© 2000 Grant Wiggins & Jay McTighe

Feedback

21

Guidance offers suggestions for improving the design, based on the designer’s stated goals (i.e., how to narrow the gap between intent & effect).

e.g., Perhaps a different Understanding would be more closely aligned with your first assessment task.© 2000 Grant Wiggins & Jay McTighe

Feedback vs. Guidance

22

Don’t Confuse Feedback with Guidance (Advice)

Too many people jump to guidance without first providing feedback and

making sure the performer understands it and agrees.

Feedback: what you did or did not do, given a standard; a neutral description of your performance or product

Guidance: what you might do to honor the feedback - good advice

23

1. What a great idea!

© 2000 Grant Wiggins & Jay McTighe

KEY: F = Feedback G = Guidance N= Neither

Check for Understanding

F G N

2. The first Essential Question is factual in nature (recall only).3. Perhaps you could begin with an experiential activity as a ‘hook’.4. I never use Word Searches in my classroom.

24

5. The mystery game is likely to ‘hook’ and engage the students.

© 2000 Grant Wiggins & Jay McTighe

KEY: F = Feedback G = Guidance N= Neither

Check for Understanding

F G N

6. We didn’t like the culminating activity.7. You might try using the ‘concept attainment’ technique here.8. We didn’t see any assessment of the second understanding.

25

Tip for Effective Peer Review

Remember that the primary goal of peer review is to provide useful feedback and guidance to help improve the design.

26

Tips for Effective Peer Review

Reviewer’s comments should be specific and “objective” - i.e. guided by and made in reference to the design standards, not personal taste.

27

Tips for Effective Peer Review

Try to understand the design and its intent before offering feedback and guidance. The designer/teacher should feel that you are trying to improve their design, not substitute your own goals or methods for theirs.

28

Feedback and its power

29

feedback and its use is key to great gainsBlack & Wiliam meta-analysis:

“There is a body of firm evidence that formative assessment is essential... We know of no other way of raising standards for which such a strong prima facie case can be made.”

Black and Wiliam (1998) “Inside the Black Box: Raising Standards through Classroom Assessment,” Phi Delta Kappan, volume 80, 2

(October), pp. 139 ff. Cf. Working Inside the Black Box: Assessment for Learning in the

Classroom, by Paul Black, Christine Harrison, Clare Lee, Bethan Marshall, and Dylan Wiliam Phi Delta Kappan, Volume 86, #1 (September, 2004)

30

Feedback: Harvard’s “most effective” coursesfrom Making the Most of College:

"The big point—it comes up over and over again as crucial—is the importance of quick and detailed feedback. Students overwhelmingly report that the single most important ingredient for making a course effective is getting rapid response on assignments and quizzes.

"Students suggest that it should be possible in certain courses to get immediate feedback. They suggest that the professor should hand out an example of an excellent answer.

- Richard Light

31

Feedback as key (cont.):

"Secondly... an overwhelming majority are convinced that their best learning takes place when they have a chance to submit an early version of their work, get detailed feedback and criticism, and then hand in a final revised version...

Many students observe that their most memorable learning experiences have come from courses where such opportunities are routine policy."

32

Feedback is vital for faculty, too

“Faculty members at Harvard were asked what single change most improved their teaching. Two ideas swamped all others. One is enhancing student awareness of the big picture, ‘the big point of it all’. The second is the importance of helpful and regular feedback from students so a professor can make midcourse corrections.”

- Harvard Assessment Seminar, 1993

33

Feedback Depends upon Models

based on: Exemplars/Models Expert Commentary on Models and non-models

Ideal specifications or performance standards

34

No gains = poor feedback system

No imrpovement over time show the problem is not “prof dvlmpt” or “staff” but the feedback system

Ponder: how would a school’s Cross Country team do if we only kept the place of finish and graded you on your relative place of finish - and did not keep your times?

35

Feedback/Self-assessment System - Healthy

Performers seek feedback on their own and know that it is in their interest - even if the news is bad

Performance improves at all levels; there is obvious “value added”

Improved performance occurs more rapidly than is typical or expected

Few quarrels about the feedback (if there are, it is about the meaning of the results)

Feedback use opportunities are central to the job

Norms and standards rise over time: what was once considered extraordinary performance becomes more common

36

Feedback/Self-assessment System - Unhealthy Learners fear, resist, do not seek, or ignore feedback

Learner performance rarely improves much beyond what is typical

Novices struggle to improve; they do not know “what you want.” Their self-assessment is very inaccurate

Many quarrels about the credibility and meaning of the results; anecdotes and effort trusted more than anything

Training is too prone to coverage and activities, with little opportunity to get feedback and use it repeatedly

Norms stay the same, standards rise - and expectations are thus lowered

37

Excellent feedback

Some criteria: Timely user-friendly - in approach and amount Descriptive & specific re: performance Consistent Expert Accurate Honest, yet constructive Derived from concrete standards On-going

38

So, what follows for staff?

How can we make feedback more common, useful,

credible - and timely? Sample their work regularly Peer review Use online resources (e.g. ubdexchange)