BADGING / MICRO-CREDENTIALING – CLASSROOM PEER REVIEW, FACILITATED AND EXTENDED BY TECHNOLOGY

“Badging” as a way to encourage review, ownership, professional-networking, lateral learning, & reflection

By: Amy McQuigge (amy.mcquigge@esc.edu) & Eileen O’Connor (eileen.oconnor@esc.edu)

SUNY CIT 2013 Empire State College

Agenda

Introduction to course where badging was implemented

Overview of the badging concept & approaches

Data and findings within the course Concluding comments / Q&A

Badging – within an online course

• Course & the student population – Learning with Emerging Technologies: Theory & Practice – Working adults take this introductory master’s courses within the

Masters of Arts in Learning & Emerging Technology • The multiple purposes for badging – (many follow-up studies

are possible for analyzing the results)– Peer review & awarding; extending course evaluation– “Lateral” learning; encouraging review & reflection; building

community & understanding – Modeling an emerging approach that these students might use in

their work• Designing learning systems that use emerging tech• Having students consider further applications

Discussion boards

Media postings w/ d-boards

Second Life – w d-board follow-up

Badge voting Blog sharing on learning

Research / resource findings via Google Drive

Final presentation /

review

Course interactions – community highly prized

course objective; a collaborative project in the

last module

The “badging” components & process within each module

The integration of the badge process• Web-accessible assignments geared to course requirements :

– additional criteria were posted for the badging;– Informal headings / descriptive / “stylistically” reinforcing the non-

graded aspect of these peer-given awards;– Additional category of creative / inventive was created for badge 4

• Postings for the class– after votes were tallied results were emailed & put in class blog– final presentation of summary badges in a virtual meeting

• The peer-voting was within Google Forms -- Data was then gathered; results were shared anonymously by instructor

• Formal presentation of badges in Second Life at end of course

Instructor review vs. peer review

Assessment points

Instructor (for credit)

Formal assessment with criteria; aligned w/ course objectives

Reports Annotated Bibliographies

Sandbox w/ technologies w/

rubrics

Peers (for badge & not credit)

5 badges – peer review opportunities; encouraging

creativity

Holistic criteria Informal categories Open ended

Example of the instructor rubric – specific & detailed; focus is on sandbox-level skills (not design or instruction

of e-mediated environment)

Badge criteria – holistic & informal; employed a deliberately different tone, language, & expectations

BADGE SCALE – Prezi / Presentation; Website; YouTube; YouTubeNo go (1) Pewter (2) Bronze (3) Gold (4)Won’t even make the grade for the assignments minimum criteria

Minimally acceptable for the assignment but nothing noteworthy in this aspect

Interesting & useful; solid display of expertise on this criteria

Wow, I am learning and taking notes here – a great job; I’ll have my friends visit here too

 

-- Used the scale above for the first 3 badges; added “willingness to try new things” to the categories above for Badge 4 (Facebook)-- Wanting to encourage students to think outside the box; some concerns that the evaluation was not “fair” to the different skill levels

All badge-prompts encouraged new areas of thinking

Created in Google Drive Forms – summarized and tallied

TOTAL COUNT OF BADGE VOTES CAST WITHIN THE 4 INDIVIDUAL AREAS

• The course asked for 3 reviews minimum; 9 students in the course; thus there should be at least 27 votes per badge

• Badge 2 – some student-participation issues

TOTAL COUNT OF BADGE VOTERS BY THE DIFFERENT AREAS

BADGE 1 BADGE 2 BADGE 3 BADGE 4

Prezi/PPT Website YouTube Facebook

51 22 33 28

Average votes per student per category (Cat 2 & 3 have multiple criteria)

Values

Row LabelsAverage of aesthetics

Average of Category2

Average of Category3

Average of “try new” category

total average

Std1 3.3 3.1 3.1 2.7 3.1Std2 3.5 3.5 3.7 3.5 3.6Std3 3.4 3.4 3.3 3.2 3.3Std4 3.2 3.2 2.8 3.0 3.0Std5 3.5 3.6 3.5 4.0 3.7Std6 3.3 3.3 2.9 2.0 2.9Std7 3.4 3.5 3.6 3.8 3.6Std8 3.4 3.1 3.3 3.0 3.2Std9 2.5 2.4 2.4 3.0 2.6

Grand Total 3.3 3.2 3.2 3.2 3.2

Std1 Std2 Std3 Std4 Std5 Std6 Std7 Std8 Std9

Average of aes

3.33333333333333

3.53846153846154

3.36842105263158

3.15384615384615

3.54545454545455

3.31578947368421

3.4375 3.42857142857143

2.54545454545455

Average of cat2

3.11111111111111

3.53846153846154

3.42105263157894

3.15384615384615

3.63636363636363

3.27777777777778

3.5 3.07142857142857

2.36363636363636

Average of cat3

3.11111111111111

3.69230769230769

3.26315789473684

2.84615384615385

3.45454545454545

2.94736842105263

3.5625 3.28571428571429

2.36363636363636

Average of try new

2.66666666666667

3.5 3.2 3 4 2 3.8 3 3

0.3

0.8

1.3

1.8

2.3

2.8

3.3

3.8

4.3

Student Average Votes per Category (several criteria in Cat 2 & 3)Av

erag

e Vo

tes (

4 hi

ghes

t val

ue)

“Try new features” category was only

added in the 4th badge event

Std1 Std2 Std3 Std4 Std5 Std6 Std7 Std8 Std9

Aver-age of aes

3.33333333333333

3.53846153846154

3.36842105263158

3.15384615384615

3.54545454545455

3.31578947368421

3.4375 3.42857142857143

2.54545454545455

Aver-age of cat2

3.11111111111111

3.53846153846154

3.42105263157894

3.15384615384615

3.63636363636363

3.27777777777778

3.5 3.07142857142857

2.36363636363636

Aver-age of cat3

3.11111111111111

3.69230769230769

3.26315789473684

2.84615384615385

3.45454545454545

2.94736842105263

3.5625 3.28571428571429

2.36363636363636

Aver-age of try new

2.66666666666667

3.5 3.2 3 4 2 3.8 3 3

0.3

0.8

1.3

1.8

2.3

2.8

3.3

3.8

4.3

Average of the votes for each student by category (cat 2 & 3 have multiple criteria) – results aligned w/ prior student

background & experience

Axis Title Std 6 & 9 were from more “traditional” institutions & environments; Std 5 & 2 had prior experience with more innovative tech

Comparison – instructor vs. peer ranking, mindful that the criteria are different for both deliberations

Instructor ranking, on “regular” course assignments & criteria

for the entire course Student # Peer ranking

1 Std8 4

2 Std2 2

2 Std3 3

2 Std6 7

3 Std5 1

3 Std9 8

4 Std4 6

4 Std7 2

5 Std1 5

Note: instructor ranking would be similar to students on the criteria considered; however, the peer assessment was on criteria beyond the course requirements

Conclusions

• Students participation was often beyond the minimum requirements;

• Students were very collegially engaged, however, the entire course reinforces this approach – further study needed to isolate the role of the badging process itself;

• Peer review mapped quite closely to instructor review & ranking;

• Using peer review (with badging) allowed for course & learning expansion in informal ways

Considerations for adaptation / further study

• Adapt concept & process to your needs – Consider how to integrate badges to encourage more examination & reflection within

your courses• You will still need to work beyond the LMS – this is extra step; however, integrating web 2.0

technologies often means extending beyond standard LMS features• Emphasize ad explain the differences between peer review vs. instructor review • Consider course objectives, audience, and the audience expectation – but don’t be afraid to

integrate and evaluate new approaches for reflection and community building

– Consider the level of granularity you need in the badges– Introduce badges early and complete a “cycle” (through to picking-up the badges)

within the course • When to issue the badges – can be associated with a course / motivational – engagement factors• For external presentation of works, perhaps

• Consider how to use badges – as rewards? as competitions? to encourage reflection & review?

• Consider when to distribute badges – during the course or at the end? • Badges can address emerging understandings about learning:

– Is professional interactions and lateral learning important? – Is additional, peer-level credentialing of value for learning within a course?