building a peer evaluation program
TRANSCRIPT
![Page 1: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/1.jpg)
Building a Peer Evaluation Program
![Page 2: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/2.jpg)
2
Housekeeping
S O ME G E N E R A L H O U S E K E E P I N G R U L E S
o A copy of the slides is available in the resources panel
o The webinar will be recorded and made available ON-Demand
o To keep the webinar short questions will be answered via email later today.
![Page 3: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/3.jpg)
Speaker
Sarah MarrsPRINCIPAL CONSULTANT, PEOPLE INSIGHTS
Sarah has dedicated her career to designing and running feedback programs for over 7 years. She joined Qualtrics in April 2015, after working in-house at Tesco overseeing their global colleague insights survey. At Qualtrics, Sarah works with clients to design feedback solutions that drive real value, through assessment design, sampling methodologies, communication programs, or interpreting results.
She’s a big believer in the power of listening to and acting on feedback, and always energised to talk about new and innovative ways to collect and act on that information.
![Page 4: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/4.jpg)
4
Agenda
TO D AY W E W I L L B E C O V E R I N G
o What is a peer evaluation programo How to create a peer evaluation program
1. Designing a competency framework and items
2. Designing your reports3. Designing your workflow
![Page 5: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/5.jpg)
5
What is peer evaluation? A peer evaluation program collects,
quantifies, and reports observations of a student by their classmates or faculty.
C O MM O N LY I N D I V I D U A L S A R E R AT E D B Y:
o Themselveso Cohort / classmateso Tutor / Professoro Potentially course directoro Potentially academic advisor
![Page 6: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/6.jpg)
6
What is peer evaluation? A peer evaluation program collects,
quantifies, and reports observations of a student by their classmates or faculty.
![Page 7: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/7.jpg)
7
What is peer evaluation? A peer evaluation program collects,
quantifies, and reports observations of a student by their classmates or faculty.
![Page 8: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/8.jpg)
8
What is peer evaluation? A peer evaluation program collects,
quantifies, and reports observations of a student by their classmates or faculty.
![Page 9: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/9.jpg)
9
What is peer evaluation? A peer evaluation program collects,
quantifies, and reports observations of a student by their classmates or faculty.
Peer evaluation is ultimately designed to drive
lasting behaviour change
![Page 10: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/10.jpg)
10
What is peer evaluation? W HAT D O S T U D E N T S G E T F R O M P E E R E VA L U AT I O N?
o Self-awareness (no.1 leadership trait!)o Improvement in learningo Teaching how to give feedbacko Teaching how to take feedback
W H AT D O E S A T U TO R G E T F R O M P E E R E VA L U AT I O N
o A feedback instrument that works for different student styleso An instrument that allows them to get a gauge on student
dynamicso An instrument that allows them to get data-driven feedback
aggregation
![Page 11: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/11.jpg)
11
What is peer evaluation? T H R E E C O M MO N U S E S F O R P E E R E VA L U AT I O N
1. Measure contributions and teamwork at the end of a specific project to feed into project grades
2. Mid-year measurement of a student’s behaviours to provide developmental feedback
3. End-of-placement review of a student work placement to get feedback from both professional and academic sides
![Page 12: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/12.jpg)
12
What is peer evaluation?
YOU MUST KNOW YOUR PEER EVALUATION PURPOSE BEFORE MOVING ON TO DESIGN YOUR PROGRAM
![Page 13: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/13.jpg)
13
How to create a peer evaluation program
T H R E E S T E P S TO C R E AT E Y O U R P E E R E VA L U AT I O N
1. Design your competency framework and items2. Design your reports3. Design your workflow
![Page 14: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/14.jpg)
14
C O MP E T E N C Y A
I T E M ( B E H AV I O U R ) 1
I T E M ( B E H AV I O U R ) 2
I T E M ( B E H AV I O U R ) 3
C O MP E T E N C Y B
C O MP E T E N C Y C
I T E M ( B E H AV I O U R ) 4
I T E M ( B E H AV I O U R ) 5
I T E M ( B E H AV I O U R ) 6
I T E M ( B E H AV I O U R ) 7
I T E M ( B E H AV I O U R ) 8
I T E M ( B E H AV I O U R ) 9
STEP 1 - Design a competency framework & items
![Page 15: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/15.jpg)
15
T E AM W O R KI N G S K I L L S
L i s t e n s t o c l a s s m a t e s
’ i d e a s
Ta k e s o n o t h e r ’s
f e e d b a c k
W o r k s w e l l w i t h
c l a s s m a t e s
C O N T R I B U T I O N TO W O R K L O A D
A t t e n d s a l l w o r k i n g s e s s i o n s
P a r t i c i p a t e s a c t i v e l y i n
w o r k i n g s e s s i o n s
I s a b l e t o c a r r y o u t t a s k s i n d e p e n d e n t l y
P L AN N I N G & O R G A N I S I N G
A r r i v e s o n t i m e t o w o r k i n g s e s s i o n s
M a n a g e s t h e i r s c h e d u l e w e l l
M e e t s p r o j e c t -r e l a t e d
d e a d l i n e s
W o r k s w e l l w i t h
c l a s s m a t e s
P R O - T I PFrequency scales are the best way to focus on the consistency of behaviours, e.g:
• Never• Rarely• Sometimes• Often• All the time
STEP 1 - Design a competency framework & items
![Page 16: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/16.jpg)
16
G O O D I T E M S . . .
o Capture an important aspect of a key competency
o Focus on a single behaviouro Start with an action verb (e.g., motivates,
listens, etc.)o Are written in simple plain languageo Can be acted upon if they are identified as
weaknesses/ opportunities
B AD I T E MS . . .
o Loosely or fail to measure an important competencyo Focus on multiple behaviours (double, triple barreled)o Include too many unnecessary adverbs (e.g.,
efficiently, effectively)o Are too technical or hard to understando Do not match the rating scaleo Are culturally biased
STEP 1 - Design a competency framework & items
![Page 17: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/17.jpg)
17
STEP 1 - Design a competency framework & items
O T H E R C O MP E T E N C Y A N D I T E M T I P S
o Avoid including too many items (generally 50-60 total items max)o Include a few (up to 3) open-ended items that ask students to provide
constructive and actionable feedback to their cohorto Remind evaluators not to include identifying information in open-ended
items to protect confidentiality
![Page 18: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/18.jpg)
18
STEP 2 - Design your reports
T W O M A I N T Y P E S O F R E P O R T
1. The individual report (to go back to the student)
2. The aggregated report (to be seen by the tutor / professor / course director only)
![Page 19: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/19.jpg)
19
STEP 2 - Design your reports
G O O D R E P O R T S :
AGGREGATE RESULTS WITHIN SOURCE – Aggregate ratings within source for each competency and item
![Page 20: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/20.jpg)
20
G O O D R E P O R T S :
AGGREGATE RESULTS WITHIN SOURCE – Aggregate ratings within source for each competency and item
PROVIDE MEANINGFUL COMPARISONS – Compare source-level ratings to internal benchmarks
STEP 2 - Design your reports
![Page 21: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/21.jpg)
21
G O O D R E P O R T S :
AGGREGATE RESULTS WITHIN SOURCE – Aggregate ratings within source for each competency and item
PROVIDE MEANINGFUL COMPARISONS – Compare source-level ratings to internal benchmarks
INCLUDE OPEN-ENDED COMMENTS – Present both positive and constructive comments
STEP 2 - Design your reports
![Page 22: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/22.jpg)
22
STEP 3 - Design your workflow C O MM O N W O RK F L O W D E C I S I O N S
o Who chooses who should evaluate an individual?o Should the tutor / professor approve nominations for
evaluators?o Should evaluators have an opportunity to opt out?o Should the individual receive their report directly?
![Page 23: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/23.jpg)
23
STEP 3 - Design your workflow S C E N A R I O 1 . MEASURE CONTR IBUT IONS AND
TEAMWORK AT THE END OF A SPEC IF IC PROJECT TO FEED INTO PROJECT GRADES
Tutor / professor adds all students in project groups
– and asks all students in group
to rate each other
All students need to complete their
project group evaluations
(no opting out)
Report goes back to tutor / professor
directly – for each individual. Aggregate report also created allowing them to
compare individuals.
Tutor discusses report with student and
feeds into project grades
![Page 24: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/24.jpg)
24
STEP 3 - Design your workflow S C E N A R I O 2 . MID-YEAR MEASUREMENT OF AN
INDIV IDUAL ’S BEHAVIOURS TO PROV IDE DEVELOPMENTAL FEEDBACK
Individual is asked to
select 3-5 peers and up
to 3 tutors who they
would like to receive
feedback from
Individual will automatically
receive evaluations from their
course director and academic advisor
Peer evaluations
are opt-in – so not
mandatory for all classmates to complete
Report goes back to
individual student first, for them to digest and
later discuss with academic
advisor or tutor.
Course directors get aggregated feedback reports
![Page 25: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/25.jpg)
25
STEP 3 - Design your workflow
Course director adds
placement students -
asking tutor, classmates
and professional coworkers to
evaluate them
Placement evaluations are ‘nudged’
but not mandatory
Individual student
report goes to each tutor for feedback discussion
Aggregate report goes to course director to
review placement
program as a whole
Course directors
reach out to company
placement teams to discuss
feedback / any red flags
S C E N A R I O 3 . END- OF-PL ACEMENT REV IEW OF A STUDENT WORK PL ACEMENT TO GET FEEDBACK FROM BOTH PROFESS IONAL AND ACADEMIC S IDES
![Page 26: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/26.jpg)
In Summary
1. Define Purpose2. Define Competencies & Behaviours3. Define Report4. Define Workflow5. Launch evaluation6. Follow-up with students
![Page 27: Building a Peer Evaluation Program](https://reader034.vdocuments.us/reader034/viewer/2022052606/58f056791a28abea728b4611/html5/thumbnails/27.jpg)
27
Thank you